Interprovider L3VPN Option C on a vMX

Implementation Description

In this article, I will build an Interprovider L3VPN Option C setup on a single vMX. Each individual router will be configured as a logical router. The vMX router has a back-to-back connection between ge-0/0/0 and ge-0/0/1 ports. For each different link, a different VLAN number will be configured to separate the traffic generated based on router ID number.

Note: This option is the most scalable solution comparing with option A and option B.

Network Diagram

IP Allocation

  • AS 20: 192.168.20.0/16
    • CE 21: 192.168.0.21/32
    • CE 22: 192.168.0.22/32
    • Link 1121: 192.168.20.0/31
    • Link 1622: 192.168.20.2/31
  • AS 30: 192.168.30.0/16
    • CE 31: 192.168.0.31/32
    • CE 32: 192.168.0.32/32
    • Link 1131: 192.168.30.0/31
    • Link 1632: 192.168.30.2/31
  • AS 100: 10.100.0.0/16
    • PE 11: 10.100.0.11/32
    • P 12: 10.100.0.12/32
    • ASBR 13: 10.100.0.13/32
    • Internal Links: 10.100.1.0/16
  • AS 200: 10.200.0.0/16
    • PE 16: 10.200.0.16/32
    • P 15: 10.200.0.15/32
    • ASBR 14: 10.200.0.14/32
    • Internal Links: 10.200.1.0/16
  • Inter-ASBR link:
    • Link 1314: 172.167.12.0/31

Full Configuration

show | no-more 
## Last changed: 2018-03-07 08:58:37 UTC
version 15.1F4.15;
groups {
    isis {
        logical-systems {
            <*> {
                protocols {
                    isis {
                        level 1 disable;
                        interface <*> {
                            point-to-point;
                        }
                    }
                }
            }
        }
    }
}
apply-groups isis;
system {
    host-name MX;
    root-authentication {
        encrypted-password "$5$L3F31155$kVyagZl2v/WM9s32/hi7VCXxM5o0vupYD.LO3uvCif4"; ## SECRET-DATA
    }
    services {
        ssh;
        netconf {
            ssh;
        }
    }
    syslog {
        user * {
            any emergency;
        }
        file messages {
            any notice;
            authorization info;
        }
        file interactive-commands {
            interactive-commands any;
        }
    }
}
logical-systems {
    11-PE {
        interfaces {
            ge-0/0/0 {
                unit 1112 {
                    vlan-id 1112;
                    family inet {
                        address 10.100.1.0/31;
                    }
                    family iso;
                    family mpls;
                }
                unit 1121 {
                    vlan-id 1121;
                    family inet {
                        address 192.168.20.0/31;
                    }
                }
                unit 1131 {
                    vlan-id 1131;
                    family inet {
                        address 192.168.30.0/31;
                    }
                }
            }
            lo0 {
                unit 11 {
                    family inet {
                        address 10.100.0.11/32;
                    }
                    family iso {
                        address 49.0100.0101.0000.0011.00;
                    }
                }
            }
        }
        protocols {
            mpls {
                interface all;
            }
            bgp {
                group to-AS200 {
                    type external;
                    multihop {
                        ttl 10;
                    }
                    local-address 10.100.0.11;
                    family inet-vpn {
                        unicast;
                    }
                    peer-as 200;
                    neighbor 10.200.0.16;
                }
                group internal {
                    type internal;
                    local-address 10.100.0.11;
                    family inet {
                        labeled-unicast {
                            resolve-vpn;
                        }
                    }
                    neighbor 10.100.0.13;
                }
            }
            isis {
                interface ge-0/0/0.1112;
                interface lo0.11;
            }
            ldp {
                interface all;
            }
        }
        routing-instances {
            Cust-20 {
                instance-type vrf;
                interface ge-0/0/0.1121;
                route-distinguisher 10.100.0.11:20;
                vrf-target target:0.0.100.200:20;
                protocols {
                    bgp {
                        group to-Cust {
                            type external;
                            peer-as 20;
                            as-override;
                            neighbor 192.168.20.1;
                        }
                    }
                }
            }
            Cust-30 {
                instance-type vrf;
                interface ge-0/0/0.1131;
                route-distinguisher 10.100.0.11:30;
                vrf-target target:0.0.100.200:30;
                protocols {
                    bgp {
                        group to-Cust {
                            type external;
                            peer-as 30;
                            as-override;
                            neighbor 192.168.30.1;
                        }
                    }
                }
            }
        }
        routing-options {
            autonomous-system 100;
        }
    }
    12-P {
        interfaces {
            ge-0/0/0 {
                unit 1213 {
                    vlan-id 1213;
                    family inet {
                        address 10.100.1.2/31;
                    }
                    family iso;
                    family mpls;
                }
            }
            ge-0/0/1 {
                unit 1112 {
                    vlan-id 1112;
                    family inet {
                        address 10.100.1.1/31;
                    }
                    family iso;
                    family mpls;
                }
            }
            lo0 {
                unit 12 {
                    family inet {
                        address 10.100.0.12/32;
                    }
                    family iso {
                        address 49.0100.0101.0000.0012.00;
                    }
                }
            }
        }
        protocols {
            mpls {
                traffic-engineering {
                    bgp-igp;
                }
                interface all;
            }
            isis {
                interface all;
            }
            ldp {
                interface all;
            }
        }
        routing-options {
            autonomous-system 100;
        }
    }
    13-ASBR {
        interfaces {
            ge-0/0/0 {
                unit 1314 {
                    vlan-id 1314;
                    family inet {
                        address 172.167.12.0/31;
                    }
                    family mpls;
                }
            }
            ge-0/0/1 {
                unit 1213 {
                    vlan-id 1213;
                    family inet {
                        address 10.100.1.3/31;
                    }
                    family iso;
                    family mpls;
                }
            }
            lo0 {
                unit 13 {
                    family inet {
                        address 10.100.0.13/32;
                    }
                    family iso {
                        address 49.0100.0101.0000.0013.00;
                    }
                }
            }
        }
        protocols {
            mpls {
                traffic-engineering {
                    bgp-igp;
                }
                interface all;
            }
            bgp {
                group to-AS200 {
                    type external;
                    family inet {
                        labeled-unicast;
                    }
                    export to-AS200;
                    peer-as 200;
                    neighbor 172.167.12.1;
                }
                group internal {
                    type internal;
                    local-address 10.100.0.13;
                    family inet {
                        labeled-unicast;
                    }
                    neighbor 10.100.0.11;
                }
            }
            isis {
                interface all;
            }
            ldp {
                interface ge-0/0/1.1213;
                interface lo0.13;
            }
        }
        policy-options {
            policy-statement to-AS200 {
                term PE11-lo0 {
                    from {
                        route-filter 10.100.0.11/32 exact;
                    }
                    then accept;
                }
                term reject {
                    then reject;
                }
            }
        }
        routing-options {
            autonomous-system 100;
        }
    }
    14-ASBR {
        interfaces {
            ge-0/0/0 {
                unit 1415 {
                    vlan-id 1415;
                    family inet {
                        address 10.200.1.4/31;
                    }
                    family iso;
                    family mpls;
                }
            }
            ge-0/0/1 {
                unit 1314 {
                    vlan-id 1314;
                    family inet {
                        address 172.167.12.1/31;
                    }
                    family mpls;
                }
            }
            lo0 {
                unit 14 {
                    family inet {
                        address 10.200.0.14/32;
                    }
                    family iso {
                        address 49.0200.0102.0000.0014.00;
                    }
                }
            }
        }
        protocols {
            mpls {
                traffic-engineering {
                    bgp-igp;
                }
                interface all;
            }
            bgp {
                group to-AS100 {
                    type external;
                    family inet {
                        labeled-unicast;
                    }
                    export to-AS100;
                    peer-as 100;
                    neighbor 172.167.12.0;
                }
                group internal {
                    type internal;
                    local-address 10.200.0.14;
                    family inet {
                        labeled-unicast;
                    }
                    neighbor 10.200.0.16;
                }
            }
            isis {
                interface all;
            }
            ldp {
                interface ge-0/0/0.1415;
                interface lo0.14;
            }
        }
        policy-options {
            policy-statement to-AS100 {
                term PE16-lo0 {
                    from {
                        route-filter 10.200.0.16/32 exact;
                    }
                    then accept;
                }
                term reject {
                    then reject;
                }
            }
        }
        routing-options {
            autonomous-system 200;
        }
    }
    15-P {
        interfaces {
            ge-0/0/0 {
                unit 1516 {
                    vlan-id 1516;
                    family inet {
                        address 10.200.1.6/31;
                    }
                    family iso;
                    family mpls;
                }
            }
            ge-0/0/1 {
                unit 1415 {
                    vlan-id 1415;
                    family inet {
                        address 10.200.1.5/31;
                    }
                    family iso;
                    family mpls;
                }
            }
            lo0 {
                unit 15 {
                    family inet {
                        address 10.200.0.15/32;
                    }
                    family iso {
                        address 49.0200.0102.0000.0015.00;
                    }
                }
            }
        }
        protocols {
            mpls {
                traffic-engineering {
                    bgp-igp;
                }
                interface all;
            }
            isis {
                interface all;
            }
            ldp {
                interface all;
            }
        }
    }
    16-PE {
        interfaces {
            ge-0/0/0 {
                unit 1622 {
                    vlan-id 1622;
                    family inet {
                        address 192.168.20.2/31;
                    }
                }
                unit 1632 {
                    vlan-id 1632;
                    family inet {
                        address 192.168.30.2/31;
                    }
                }
            }
            ge-0/0/1 {
                unit 1516 {
                    vlan-id 1516;
                    family inet {
                        address 10.200.1.7/31;
                    }
                    family iso;
                    family mpls;
                }
            }
            lo0 {
                unit 16 {
                    family inet {
                        address 10.200.0.16/32;
                    }
                    family iso {
                        address 49.0200.0102.0000.0016.00;
                    }
                }
            }
        }
        protocols {
            mpls {
                interface all;
            }
            bgp {
                group internal {
                    type internal;
                    local-address 10.200.0.16;
                    family inet {
                        labeled-unicast {
                            resolve-vpn;
                        }
                    }
                    neighbor 10.200.0.14;
                }
                group to-AS100 {
                    type external;
                    multihop {
                        ttl 10;
                    }
                    local-address 10.200.0.16;
                    family inet-vpn {
                        unicast;
                    }
                    peer-as 100;
                    neighbor 10.100.0.11;
                }
            }
            isis {
                interface all;
            }
            ldp {
                interface all;
            }
        }
        routing-instances {
            Cust-20 {
                instance-type vrf;
                interface ge-0/0/0.1622;
                route-distinguisher 10.100.0.16:20;
                vrf-target target:0.0.100.200:20;
                protocols {
                    bgp {
                        group to-Cust {
                            type external;
                            peer-as 20;
                            as-override;
                            neighbor 192.168.20.3;
                        }
                    }
                }
            }
            Cust-30 {
                instance-type vrf;
                interface ge-0/0/0.1632;
                route-distinguisher 10.100.0.16:30;
                vrf-target target:0.0.100.200:30;
                protocols {
                    bgp {
                        group to-Cust {
                            type external;
                            peer-as 30;
                            as-override;
                            neighbor 192.168.30.3;
                        }
                    }
                }
            }
        }
        routing-options {
            autonomous-system 200;
        }
    }
    21-CE {
        interfaces {
            ge-0/0/1 {
                unit 1121 {
                    vlan-id 1121;
                    family inet {
                        address 192.168.20.1/31;
                    }
                }
            }
            lo0 {
                unit 21 {
                    family inet {
                        address 192.168.0.21/32;
                    }
                }
            }
        }
        protocols {
            bgp {
                group AS100 {
                    type external;
                    export to-bgp;
                    peer-as 100;
                    neighbor 192.168.20.0;
                }
            }
        }
        policy-options {
            policy-statement to-bgp {
                from protocol direct;
                then accept;
            }
        }
        routing-options {
            autonomous-system 20;
        }
    }
    22-CE {
        interfaces {
            ge-0/0/1 {
                unit 1622 {
                    vlan-id 1622;
                    family inet {
                        address 192.168.20.3/31;
                    }
                }
            }
            lo0 {
                unit 22 {
                    family inet {
                        address 192.168.0.22/32;
                    }
                }
            }
        }
        protocols {
            bgp {
                group AS200 {
                    type external;
                    export to-bgp;
                    peer-as 200;
                    neighbor 192.168.20.2;
                }
            }
        }
        policy-options {
            policy-statement to-bgp {
                from protocol direct;
                then accept;
            }
        }
        routing-options {
            autonomous-system 20;
        }
    }
    31-CE {
        interfaces {
            ge-0/0/1 {
                unit 1131 {
                    vlan-id 1131;
                    family inet {
                        address 192.168.30.1/31;
                    }
                }
            }
            lo0 {
                unit 31 {
                    family inet {
                        address 192.168.0.31/32;
                    }
                }
            }
        }
        protocols {
            bgp {
                group AS100 {
                    type external;
                    export to-bgp;
                    peer-as 100;
                    neighbor 192.168.30.0;
                }
            }
        }
        policy-options {
            policy-statement to-bgp {
                from protocol direct;
                then accept;
            }
        }
        routing-options {
            autonomous-system 30;
        }
    }
    32-CE {
        interfaces {
            ge-0/0/1 {
                unit 1632 {
                    vlan-id 1632;
                    family inet {
                        address 192.168.30.3/31;
                    }
                }
            }
            lo0 {
                unit 32 {
                    family inet {
                        address 192.168.0.32/32;
                    }
                }
            }
        }
        protocols {
            bgp {
                group AS200 {
                    type external;
                    export to-bgp;
                    peer-as 200;
                    neighbor 192.168.30.2;
                }
            }
        }
        policy-options {
            policy-statement to-bgp {
                from protocol direct;
                then accept;
            }
        }
        routing-options {
            autonomous-system 30;
        }
    }
}
interfaces {
    ge-0/0/0 {
        description "to ge-0/0/1";
        vlan-tagging;
    }
    ge-0/0/1 {
        description "to ge-0/0/0";
        vlan-tagging;
    }
    ge-0/0/2 {
        description "to ge-0/0/3";
        vlan-tagging;
    }
    ge-0/0/3 {
        description "to ge-0/0/2";
        vlan-tagging;
    }
    ge-0/0/4 {
        description "to ge-0/0/5";
        vlan-tagging;
    }
    ge-0/0/5 {
        description "to ge-0/0/4";
        vlan-tagging;
    }
    ge-0/0/6 {
        description "to ge-0/0/7";
        vlan-tagging;
    }
    ge-0/0/7 {
        description "to ge-0/0/6";
        vlan-tagging;
    }
    ge-0/0/8 {
        description "to eth1";
    }
    ge-0/0/9 {
        description "to eth2";
    }
    fxp0 {
        unit 0 {
            family inet {
                address 192.168.83.11/24;
            }
        }
    }
}

Verification

root@MX:21-CE> show route 

inet.0: 5 destinations, 5 routes (5 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

192.168.0.21/32    *[Direct/0] 02:05:23
                    > via lo0.21
192.168.0.22/32    *[BGP/170] 00:00:33, localpref 100
                      AS path: 100 200 100 I, validation-state: unverified
                    > to 192.168.20.0 via ge-0/0/1.1121
192.168.20.0/31    *[Direct/0] 02:04:45
                    > via ge-0/0/1.1121
192.168.20.1/32    *[Local/0] 02:04:45
                      Local via ge-0/0/1.1121
192.168.20.2/31    *[BGP/170] 00:00:33, localpref 100
                      AS path: 100 200 I, validation-state: unverified
                    > to 192.168.20.0 via ge-0/0/1.1121

root@MX:21-CE> ping 192.168.0.22 count 3 source 192.168.0.21    
PING 192.168.0.22 (192.168.0.22): 56 data bytes
64 bytes from 192.168.0.22: icmp_seq=0 ttl=52 time=11.834 ms
64 bytes from 192.168.0.22: icmp_seq=1 ttl=52 time=11.269 ms
64 bytes from 192.168.0.22: icmp_seq=2 ttl=52 time=8.738 ms

--- 192.168.0.22 ping statistics ---
3 packets transmitted, 3 packets received, 0% packet loss
round-trip min/avg/max/stddev = 8.738/10.614/11.834/1.346 ms

Label Operation

1. Customer router will send simple traffic.

root@MX:21-CE> show route 192.168.0.22 

inet.0: 5 destinations, 5 routes (5 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

192.168.0.22/32    *[BGP/170] 00:05:33, localpref 100
                      AS path: 100 200 100 I, validation-state: unverified
                    > to 192.168.20.0 via ge-0/0/1.1121

2. Router 11-PE will add three labels to the traffic, L3VPN label, BGP-LU label and LDP transport label.

root@MX:11-PE> show route 192.168.0.22    

Cust-20.inet.0: 5 destinations, 6 routes (5 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

192.168.0.22/32    *[BGP/170] 00:10:04, localpref 100, from 10.200.0.16
                      AS path: 200 20 I, validation-state: unverified
                    > to 10.100.1.1 via ge-0/0/0.1112, Push 299856, Push 299952, Push 299824(top)

root@MX:11-PE> show route table mpls      

mpls.0: 9 destinations, 9 routes (9 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

0                  *[MPLS/0] 02:15:03, metric 1
                      Receive
1                  *[MPLS/0] 02:15:03, metric 1
                      Receive
2                  *[MPLS/0] 02:15:03, metric 1
                      Receive
13                 *[MPLS/0] 02:15:03, metric 1
                      Receive
299776             *[LDP/9] 02:14:03, metric 1
                    > to 10.100.1.1 via ge-0/0/0.1112, Pop      
299776(S=0)        *[LDP/9] 02:14:03, metric 1
                    > to 10.100.1.1 via ge-0/0/0.1112, Pop      
299840             *[LDP/9] 00:30:25, metric 1
                    > to 10.100.1.1 via ge-0/0/0.1112, Swap 299824
299888             *[VPN/170] 00:10:08
                    > to 192.168.20.1 via ge-0/0/0.1121, Pop      
299904             *[VPN/170] 00:10:08
                    > to 192.168.30.1 via ge-0/0/0.1131, Pop           

3. Router 12-P is PHP and will pop the LDP label.

root@MX:12-P> show route 192.168.0.22  

root@MX:12-P> show route label 299824 

mpls.0: 8 destinations, 8 routes (8 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

299824             *[LDP/9] 00:31:02, metric 1
                    > to 10.100.1.3 via ge-0/0/0.1213, Pop      
299824(S=0)        *[LDP/9] 00:31:02, metric 1
                    > to 10.100.1.3 via ge-0/0/0.1213, Pop         

4. Router 13-ASBR will swap the BGP-LU label.

root@MX:13-ASBR> show route label 299952 

mpls.0: 9 destinations, 9 routes (9 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

299952             *[VPN/170] 00:15:19
                    > to 172.167.12.1 via ge-0/0/0.1314, Swap 299920

5. Router 14-ASBR will swap BGP-LU with a new LDP label.

root@MX:14-ASBR> show route 10.200.0.16 

inet.0: 10 destinations, 12 routes (10 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

10.200.0.16/32     *[LDP/9] 00:30:22, metric 1
                    > to 10.200.1.5 via ge-0/0/0.1415, Push 299824
                    [IS-IS/18] 00:31:56, metric 20
                    > to 10.200.1.5 via ge-0/0/0.1415

root@MX:14-ASBR> show route label 299920    

mpls.0: 9 destinations, 9 routes (9 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

299920             *[VPN/170] 00:27:37
                    > to 10.200.1.5 via ge-0/0/0.1415, Swap 299824

6. Router 15-P is PHP and will pop the LDP label.

root@MX:15-P> show route 192.168.0.22 

root@MX:15-P> show route label 299824 

mpls.0: 8 destinations, 8 routes (8 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

299824             *[LDP/9] 00:31:21, metric 1
                    > to 10.200.1.7 via ge-0/0/0.1516, Pop      
299824(S=0)        *[LDP/9] 00:31:21, metric 1
                    > to 10.200.1.7 via ge-0/0/0.1516, Pop

7. Router 16-PE will pop the L3VPN label and will forward simple traffic to the end customer.

root@MX:16-PE> show route label 299856 

mpls.0: 9 destinations, 9 routes (9 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

299856             *[VPN/170] 00:20:01
                    > to 192.168.20.3 via ge-0/0/0.1622, Pop      

root@MX:16-PE> show route 192.168.0.22 

Cust-20.inet.0: 5 destinations, 6 routes (5 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

192.168.0.22/32    *[BGP/170] 02:24:35, localpref 100
                      AS path: 20 I, validation-state: unverified
                    > to 192.168.20.3 via ge-0/0/0.1622

Sources:

Interprovider L3VPN Option B on a vMX

Implementation Description

In this article, I will build an Interprovider L3VPN Option B setup on a single vMX. Each individual router will be configured as a logical router. The vMX router has a back-to-back connection between ge-0/0/0 and ge-0/0/1 ports. For each different link, a different VLAN number will be configured to separate the traffic generated based on router ID number.

Network Diagram

IP Allocation

  • AS 20: 192.168.20.0/16
    • CE 21: 192.168.0.21/32
    • CE 22: 192.168.0.22/32
    • Link 1121: 192.168.20.0/31
    • Link 1622: 192.168.20.2/31
  • AS 30: 192.168.30.0/16
    • CE 31: 192.168.0.31/32
    • CE 32: 192.168.0.32/32
    • Link 1131: 192.168.30.0/31
    • Link 1632: 192.168.30.2/31
  • AS 100: 10.100.0.0/16
    • PE 11: 10.100.0.11/32
    • P 12: 10.100.0.12/32
    • ASBR 13: 10.100.0.13/32
    • Internal Links: 10.100.1.0/16
  • AS 200: 10.200.0.0/16
    • PE 16: 10.200.0.16/32
    • P 15: 10.200.0.15/32
    • ASBR 14: 10.200.0.14/32
    • Internal Links: 10.200.1.0/16
  • Inter-ASBR link:
    • Link 1314: 172.167.12.0/31

Full Configuration

## Last changed: 2018-03-07 07:03:07 UTC
version 15.1F4.15;
groups {
    isis {
        logical-systems {
            <*> {
                protocols {
                    isis {
                        level 1 disable;
                        interface <*> {
                            point-to-point;
                        }
                    }
                }
            }
        }
    }
}
apply-groups isis;
system {
    host-name MX;
    root-authentication {
        encrypted-password "$5$L3F31155$kVyagZl2v/WM9s32/hi7VCXxM5o0vupYD.LO3uvCif4"; ## SECRET-DATA
    }
    services {
        ssh;
        netconf {
            ssh;
        }
    }
    syslog {
        user * {
            any emergency;
        }
        file messages {
            any notice;
            authorization info;
        }
        file interactive-commands {
            interactive-commands any;
        }
    }
}
logical-systems {
    11-PE {
        interfaces {
            ge-0/0/0 {
                unit 1112 {
                    vlan-id 1112;
                    family inet {
                        address 10.100.1.0/31;
                    }
                    family iso;
                    family mpls;
                }
                unit 1121 {
                    vlan-id 1121;
                    family inet {
                        address 192.168.20.0/31;
                    }
                }
                unit 1131 {
                    vlan-id 1131;
                    family inet {
                        address 192.168.30.0/31;
                    }
                }
            }
            lo0 {
                unit 11 {
                    family inet {
                        address 10.100.0.11/32;
                    }
                    family iso {
                        address 49.0100.0101.0000.0011.00;
                    }
                }
            }
        }
        protocols {
            mpls {
                interface all;
            }
            bgp {
                group internal {
                    type internal;
                    local-address 10.100.0.11;
                    family inet-vpn {
                        any;
                    }
                    neighbor 10.100.0.13;
                }
            }
            isis {
                interface ge-0/0/0.1112;
                interface lo0.11;
            }
            ldp {
                interface all;
            }
        }
        routing-instances {
            Cust-20 {
                instance-type vrf;
                interface ge-0/0/0.1121;
                route-distinguisher 10.100.0.11:20;
                vrf-target target:0.0.100.200:20;
                protocols {
                    bgp {
                        group to-Cust {
                            type external;
                            peer-as 20;
                            as-override;
                            neighbor 192.168.20.1;
                        }
                    }
                }
            }
            Cust-30 {
                instance-type vrf;
                interface ge-0/0/0.1131;
                route-distinguisher 10.100.0.11:30;
                vrf-target target:0.0.100.200:30;
                protocols {
                    bgp {
                        group to-Cust {
                            type external;
                            peer-as 30;
                            as-override;
                            neighbor 192.168.30.1;
                        }
                    }
                }
            }
        }
        routing-options {
            autonomous-system 100;
        }
    }
    12-P {
        interfaces {
            ge-0/0/0 {
                unit 1213 {
                    vlan-id 1213;
                    family inet {
                        address 10.100.1.2/31;
                    }
                    family iso;
                    family mpls;
                }
            }
            ge-0/0/1 {
                unit 1112 {
                    vlan-id 1112;
                    family inet {
                        address 10.100.1.1/31;
                    }
                    family iso;
                    family mpls;
                }
            }
            lo0 {
                unit 12 {
                    family inet {
                        address 10.100.0.12/32;
                    }
                    family iso {
                        address 49.0100.0101.0000.0012.00;
                    }
                }
            }
        }
        protocols {
            mpls {
                interface all;
            }
            isis {
                interface all;
            }
            ldp {
                interface all;
            }
        }
        routing-options {
            autonomous-system 100;
        }
    }
    13-ASBR {
        interfaces {
            ge-0/0/0 {
                unit 1314 {
                    vlan-id 1314;
                    family inet {
                        address 172.167.12.0/31;
                    }
                    family mpls;
                }
            }
            ge-0/0/1 {
                unit 1213 {
                    vlan-id 1213;
                    family inet {
                        address 10.100.1.3/31;
                    }
                    family iso;
                    family mpls;
                }
            }
            lo0 {
                unit 13 {
                    family inet {
                        address 10.100.0.13/32;
                    }
                    family iso {
                        address 49.0100.0101.0000.0013.00;
                    }
                }
            }
        }
        protocols {
            mpls {
                interface all;
            }
            bgp {
                group internal {
                    type internal;
                    local-address 10.100.0.13;
                    family inet-vpn {
                        any;
                    }
                    neighbor 10.100.0.11;
                }
                group to-AS200 {
                    type external;
                    family inet-vpn {
                        any;
                    }
                    peer-as 200;
                    neighbor 172.167.12.1;
                }
            }
            isis {
                interface all;
            }
            ldp {
                interface ge-0/0/1.1213;
                interface lo0.13;
            }
        }
        routing-options {
            autonomous-system 100;
        }
    }
    14-ASBR {
        interfaces {
            ge-0/0/0 {
                unit 1415 {
                    vlan-id 1415;
                    family inet {
                        address 10.200.1.4/31;
                    }
                    family iso;
                    family mpls;
                }
            }
            ge-0/0/1 {
                unit 1314 {
                    vlan-id 1314;
                    family inet {
                        address 172.167.12.1/31;
                    }
                    family mpls;
                }
            }
            lo0 {
                unit 14 {
                    family inet {
                        address 10.200.0.14/32;
                    }
                    family iso {
                        address 49.0200.0102.0000.0014.00;
                    }
                }
            }
        }
        protocols {
            mpls {
                interface all;
            }
            bgp {
                group internal {
                    type internal;
                    local-address 10.200.0.14;
                    family inet-vpn {
                        any;
                    }
                    neighbor 10.200.0.16;
                }
                group to-AS100 {
                    type external;
                    family inet-vpn {
                        any;
                    }
                    peer-as 100;
                    neighbor 172.167.12.0;
                }
            }
            isis {
                interface all;
            }
            ldp {
                interface ge-0/0/0.1415;
                interface lo0.14;
            }
        }
        routing-options {
            autonomous-system 200;
        }
    }
    15-P {
        interfaces {
            ge-0/0/0 {
                unit 1516 {
                    vlan-id 1516;
                    family inet {
                        address 10.200.1.6/31;
                    }
                    family iso;
                    family mpls;
                }
            }
            ge-0/0/1 {
                unit 1415 {
                    vlan-id 1415;
                    family inet {
                        address 10.200.1.5/31;
                    }
                    family iso;
                    family mpls;
                }
            }
            lo0 {
                unit 15 {
                    family inet {
                        address 10.200.0.15/32;
                    }
                    family iso {
                        address 49.0200.0102.0000.0015.00;
                    }
                }
            }
        }
        protocols {
            mpls {
                interface all;
            }
            isis {
                interface all;
            }
            ldp {
                interface all;
            }
        }
    }
    16-PE {
        interfaces {
            ge-0/0/0 {
                unit 1622 {
                    vlan-id 1622;
                    family inet {
                        address 192.168.20.2/31;
                    }
                }
                unit 1632 {
                    vlan-id 1632;
                    family inet {
                        address 192.168.30.2/31;
                    }
                }
            }
            ge-0/0/1 {
                unit 1516 {
                    vlan-id 1516;
                    family inet {
                        address 10.200.1.7/31;
                    }
                    family iso;
                    family mpls;
                }
            }
            lo0 {
                unit 16 {
                    family inet {
                        address 10.200.0.16/32;
                    }
                    family iso {
                        address 49.0200.0102.0000.0016.00;
                    }
                }
            }
        }
        protocols {
            mpls {
                interface all;
            }
            bgp {
                group internal {
                    type internal;
                    local-address 10.200.0.16;
                    family inet-vpn {
                        any;
                    }
                    neighbor 10.200.0.14;
                }
            }
            isis {
                interface all;
            }
            ldp {
                interface all;
            }
        }
        routing-instances {
            Cust-20 {
                instance-type vrf;
                interface ge-0/0/0.1622;
                route-distinguisher 10.100.0.16:20;
                vrf-target target:0.0.100.200:20;
                protocols {
                    bgp {
                        group to-Cust {
                            type external;
                            peer-as 20;
                            as-override;
                            neighbor 192.168.20.3;
                        }
                    }
                }
            }
            Cust-30 {
                instance-type vrf;
                interface ge-0/0/0.1632;
                route-distinguisher 10.100.0.16:30;
                vrf-target target:0.0.100.200:30;
                protocols {
                    bgp {
                        group to-Cust {
                            type external;
                            peer-as 30;
                            as-override;
                            neighbor 192.168.30.3;
                        }
                    }
                }
            }
        }
        routing-options {
            autonomous-system 200;
        }
    }
    21-CE {
        interfaces {
            ge-0/0/1 {
                unit 1121 {
                    vlan-id 1121;
                    family inet {
                        address 192.168.20.1/31;
                    }
                }
            }
            lo0 {
                unit 21 {
                    family inet {
                        address 192.168.0.21/32;
                    }
                }
            }
        }
        protocols {
            bgp {
                group AS100 {
                    type external;
                    export to-bgp;
                    peer-as 100;
                    neighbor 192.168.20.0;
                }
            }
        }
        policy-options {
            policy-statement to-bgp {
                from protocol direct;
                then accept;
            }
        }
        routing-options {
            autonomous-system 20;
        }
    }
    22-CE {
        interfaces {
            ge-0/0/1 {
                unit 1622 {
                    vlan-id 1622;
                    family inet {
                        address 192.168.20.3/31;
                    }
                }
            }
            lo0 {
                unit 22 {
                    family inet {
                        address 192.168.0.22/32;
                    }
                }
            }
        }
        protocols {
            bgp {
                group AS200 {
                    type external;
                    export to-bgp;
                    peer-as 200;
                    neighbor 192.168.20.2;
                }
            }
        }
        policy-options {
            policy-statement to-bgp {
                from protocol direct;
                then accept;
            }
        }
        routing-options {
            autonomous-system 20;
        }
    }
    31-CE {
        interfaces {
            ge-0/0/1 {
                unit 1131 {
                    vlan-id 1131;
                    family inet {
                        address 192.168.30.1/31;
                    }
                }
            }
            lo0 {
                unit 31 {
                    family inet {
                        address 192.168.0.31/32;
                    }
                }
            }
        }
        protocols {
            bgp {
                group AS100 {
                    type external;
                    export to-bgp;
                    peer-as 100;
                    neighbor 192.168.30.0;
                }
            }
        }
        policy-options {
            policy-statement to-bgp {
                from protocol direct;
                then accept;
            }
        }
        routing-options {
            autonomous-system 30;
        }
    }
    32-CE {
        interfaces {
            ge-0/0/1 {
                unit 1632 {
                    vlan-id 1632;
                    family inet {
                        address 192.168.30.3/31;
                    }
                }
            }
            lo0 {
                unit 32 {
                    family inet {
                        address 192.168.0.32/32;
                    }
                }
            }
        }
        protocols {
            bgp {
                group AS200 {
                    type external;
                    export to-bgp;
                    peer-as 200;
                    neighbor 192.168.30.2;
                }
            }
        }
        policy-options {
            policy-statement to-bgp {
                from protocol direct;
                then accept;
            }
        }
        routing-options {
            autonomous-system 30;
        }
    }
}
interfaces {
    ge-0/0/0 {
        description "to ge-0/0/1";
        vlan-tagging;
    }
    ge-0/0/1 {
        description "to ge-0/0/0";
        vlan-tagging;
    }
    ge-0/0/2 {
        description "to ge-0/0/3";
        vlan-tagging;
    }
    ge-0/0/3 {
        description "to ge-0/0/2";
        vlan-tagging;
    }
    ge-0/0/4 {
        description "to ge-0/0/5";
        vlan-tagging;
    }
    ge-0/0/5 {
        description "to ge-0/0/4";
        vlan-tagging;
    }
    ge-0/0/6 {
        description "to ge-0/0/7";
        vlan-tagging;
    }
    ge-0/0/7 {
        description "to ge-0/0/6";
        vlan-tagging;
    }
    ge-0/0/8 {
        description "to eth1";
    }
    ge-0/0/9 {
        description "to eth2";
    }
    fxp0 {
        unit 0 {
            family inet {
                address 192.168.83.11/24;
            }
        }
    }
}

Verification

root@MX:21-CE> show route    

inet.0: 5 destinations, 5 routes (5 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

192.168.0.21/32    *[Direct/0] 00:09:22
                    > via lo0.21
192.168.0.22/32    *[BGP/170] 00:00:14, localpref 100
                      AS path: 100 200 100 I, validation-state: unverified
                    > to 192.168.20.0 via ge-0/0/1.1121
192.168.20.0/31    *[Direct/0] 00:08:44
                    > via ge-0/0/1.1121
192.168.20.1/32    *[Local/0] 00:08:44
                      Local via ge-0/0/1.1121
192.168.20.2/31    *[BGP/170] 00:00:13, localpref 100
                      AS path: 100 200 I, validation-state: unverified
                    > to 192.168.20.0 via ge-0/0/1.1121

root@MX:21-CE> ping 192.168.0.22 count 3 source 192.168.0.21  
PING 192.168.0.22 (192.168.0.22): 56 data bytes
64 bytes from 192.168.0.22: icmp_seq=0 ttl=52 time=11.001 ms
64 bytes from 192.168.0.22: icmp_seq=1 ttl=52 time=11.744 ms
64 bytes from 192.168.0.22: icmp_seq=2 ttl=52 time=12.737 ms

--- 192.168.0.22 ping statistics ---
3 packets transmitted, 3 packets received, 0% packet loss
round-trip min/avg/max/stddev = 11.001/11.827/12.737/0.711 ms

Label Operation

1. Customer router will send simple traffic.

root@MX:21-CE> show route 192.168.0.22 

inet.0: 5 destinations, 5 routes (5 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

192.168.0.22/32    *[BGP/170] 00:05:25, localpref 100
                      AS path: 100 200 100 I, validation-state: unverified
                    > to 192.168.20.0 via ge-0/0/1.1121

2. Router 11-PE will add two labels to the traffic, L3VPN label and LDP transport label.

root@MX:11-PE> show route 192.168.0.22 

Cust-20.inet.0: 5 destinations, 6 routes (5 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

192.168.0.22/32    *[BGP/170] 00:06:06, localpref 100, from 10.100.0.13
                      AS path: 200 20 I, validation-state: unverified
                    > to 10.100.1.1 via ge-0/0/0.1112, Push 299840, Push 299792(top)

root@MX:11-PE> show route table mpls 

mpls.0: 9 destinations, 9 routes (9 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

0                  *[MPLS/0] 00:16:38, metric 1
                      Receive
1                  *[MPLS/0] 00:16:38, metric 1
                      Receive
2                  *[MPLS/0] 00:16:38, metric 1
                      Receive
13                 *[MPLS/0] 00:16:38, metric 1
                      Receive
299776             *[LDP/9] 00:15:38, metric 1
                    > to 10.100.1.1 via ge-0/0/0.1112, Pop      
299776(S=0)        *[LDP/9] 00:15:38, metric 1
                    > to 10.100.1.1 via ge-0/0/0.1112, Pop      
299792             *[LDP/9] 00:15:13, metric 1
                    > to 10.100.1.1 via ge-0/0/0.1112, Swap 299792
299808             *[VPN/170] 00:14:21
                    > to 192.168.20.1 via ge-0/0/0.1121, Pop      
299824             *[VPN/170] 00:14:21
                    > to 192.168.30.1 via ge-0/0/0.1131, Pop      

3. Router 12-P is PHP and will pop the LDP label.

root@MX:12-P> show route 192.168.0.22  

root@MX:12-P> show route label 299792 

root@MX:12-P> show route label 299792 

mpls.0: 8 destinations, 8 routes (8 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

299792             *[LDP/9] 00:16:25, metric 1
                    > to 10.100.1.3 via ge-0/0/0.1213, Pop      
299792(S=0)        *[LDP/9] 00:16:25, metric 1
                    > to 10.100.1.3 via ge-0/0/0.1213, Pop      

4. Router 13-ASBR will swap the L3VPN label.

root@MX:13-ASBR> show route table bgp.l3vpn.0                                  

bgp.l3vpn.0: 8 destinations, 8 routes (8 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

[...]
10.100.0.16:20:192.168.0.22/32                
                   *[BGP/170] 00:11:44, localpref 100
                      AS path: 200 20 I, validation-state: unverified
                    > to 172.167.12.1 via ge-0/0/0.1314, Push 299808
[...]


root@MX:13-ASBR> show route label 299840 

mpls.0: 11 destinations, 11 routes (11 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

299840             *[VPN/170] 00:21:11
                    > to 172.167.12.1 via ge-0/0/0.1314, Swap 299808

5. Router 14-ASBR will swap L3VPN label and will push LDP label.

root@MX:14-ASBR> show route table bgp.l3vpn.0 

bgp.l3vpn.0: 8 destinations, 8 routes (8 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

[...]
10.100.0.16:20:192.168.0.22/32                
                   *[BGP/170] 00:20:30, localpref 100, from 10.200.0.16
                      AS path: 20 I, validation-state: unverified
                    > to 10.200.1.5 via ge-0/0/0.1415, Push 299808, Push 299776(top)
[...]

root@MX:14-ASBR> show route label 299808                     

mpls.0: 11 destinations, 11 routes (11 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

299808             *[VPN/170] 00:27:53, metric2 1, from 10.200.0.16
                    > to 10.200.1.5 via ge-0/0/0.1415, Swap 299808, Push 299776(top)

6. Router 15-P is PHP and will pop the LDP label.

root@MX:15-P> show route 192.168.0.22 

root@MX:15-P> show route label 299776    

mpls.0: 8 destinations, 8 routes (8 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

299776             *[LDP/9] 00:33:15, metric 1
                    > to 10.200.1.7 via ge-0/0/0.1516, Pop      
299776(S=0)        *[LDP/9] 00:33:15, metric 1
                    > to 10.200.1.7 via ge-0/0/0.1516, Pop 

7. Router 16-PE will pop L3VPN label and will forward simple traffic to the end customer.

root@MX:16-PE> show route label 299808 

mpls.0: 9 destinations, 9 routes (9 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

299808             *[VPN/170] 00:33:13
                    > to 192.168.20.3 via ge-0/0/0.1622, Pop      

root@MX:16-PE> show route 192.168.0.22    

Cust-20.inet.0: 5 destinations, 6 routes (5 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

192.168.0.22/32    *[BGP/170] 00:34:52, localpref 100
                      AS path: 20 I, validation-state: unverified
                    > to 192.168.20.3 via ge-0/0/0.1622

The Factors that Limit the Scalability of this Method

  • In this solution, ASBR routers keep all VPN-IPv4 routes in the routing information base (RIB), and the labels associated with the prefixes are kept in the forwarding information base (FIB). Because the RIB and FIB tables can take too much of the respective allocated memory, this solution is not very scalable for an interprovider VPN.
  • If a transit SP is used between SP1 and SP2, the transit SP also has to keep all VPN-IPv4 routes in the RIB and the corresponding labels in the FIB. The ASBRs at the transit SP have the same functionality as ASBRs at SP1 or SP2 in this solution.

Sources:

 

Interprovider L3VPN Option A on a vMX

Implementation Description

In this article, I will build an Interprovider L3VPN Option A setup on a single vMX. Each individual router will be configured as a logical router. The vMX router has a back-to-back connection between ge-0/0/0 and ge-0/0/1 ports. For each different link, a different VLAN number will be configured to separate the traffic generated based on router ID number.

Network Diagram

IP Allocation

  • AS 20: 192.168.20.0/16
    • CE 21: 192.168.0.21/32
    • CE 22: 192.168.0.22/32
    • Link 1121: 192.168.20.0/31
    • Link 1622: 192.168.20.2/31
  • AS 30: 192.168.30.0/16
    • CE 31: 192.168.0.31/32
    • CE 32: 192.168.0.32/32
    • Link 1131: 192.168.30.0/31
    • Link 1632: 192.168.30.2/31
  • AS 100: 10.100.0.0/16
    • PE 11: 10.100.0.11/32
    • P 12: 10.100.0.12/32
    • ASBR 13: 10.100.0.13/32
    • Internal Links: 10.100.1.0/16
  • AS 200: 10.200.0.0/16
    • PE 16: 10.200.0.16/32
    • P 15: 10.200.0.15/32
    • ASBR 14: 10.200.0.14/32
    • Internal Links: 10.200.1.0/16
  • Inter-ASBR links:
    • Link 20: 172.167.20.0/31
    • LInk 30: 172.167.30.0/31

Full Configuration

## Last changed: 2018-03-06 12:52:17 UTC
version 15.1F4.15;
groups {
    isis {
        logical-systems {
            <*> {
                protocols {
                    isis {
                        level 1 disable;
                        interface <*> {
                            point-to-point;
                        }
                    }
                }
            }
        }
    }
}
apply-groups isis;
system {
    host-name MX;
    root-authentication {
        encrypted-password "$5$L3F31155$kVyagZl2v/WM9s32/hi7VCXxM5o0vupYD.LO3uvCif4"; ## SECRET-DATA
    }
    services {
        ssh;
        netconf {
            ssh;
        }
    }
    syslog {
        user * {
            any emergency;
        }
        file messages {
            any notice;
            authorization info;
        }
        file interactive-commands {
            interactive-commands any;
        }
    }
}
logical-systems {
    11-PE {
        interfaces {
            ge-0/0/0 {
                unit 1112 {
                    vlan-id 1112;
                    family inet {
                        address 10.100.1.0/31;
                    }
                    family iso;
                    family mpls;
                }
                unit 1121 {
                    vlan-id 1121;
                    family inet {
                        address 192.168.20.0/31;
                    }
                }
                unit 1131 {
                    vlan-id 1131;
                    family inet {
                        address 192.168.30.0/31;
                    }
                }
            }
            lo0 {
                unit 11 {
                    family inet {
                        address 10.100.0.11/32;
                    }
                    family iso {
                        address 49.0100.0101.0000.0011.00;
                    }
                }
            }
        }
        protocols {
            mpls {
                interface all;
            }
            bgp {
                group internal {
                    type internal;
                    local-address 10.100.0.11;
                    family inet-vpn {
                        any;
                    }
                    neighbor 10.100.0.13;
                }
            }
            isis {
                interface ge-0/0/0.1112;
                interface lo0.11;
            }
            ldp {
                interface all;
            }
        }
        routing-instances {
            Cust-20 {
                instance-type vrf;
                interface ge-0/0/0.1121;
                route-distinguisher 10.100.0.11:20;
                vrf-target target:100:20;
                protocols {
                    bgp {
                        group to-Cust {
                            type external;
                            peer-as 20;
                            as-override;
                            neighbor 192.168.20.1;
                        }
                    }
                }
            }
            Cust-30 {
                instance-type vrf;
                interface ge-0/0/0.1131;
                route-distinguisher 10.100.0.11:30;
                vrf-target target:100:30;
                protocols {
                    bgp {
                        group to-Cust {
                            type external;
                            peer-as 30;
                            as-override;
                            neighbor 192.168.30.1;
                        }
                    }
                }
            }
        }
        routing-options {
            autonomous-system 100;
        }
    }
    12-P {
        interfaces {
            ge-0/0/0 {
                unit 1213 {
                    vlan-id 1213;
                    family inet {
                        address 10.100.1.2/31;
                    }
                    family iso;
                    family mpls;
                }
            }
            ge-0/0/1 {
                unit 1112 {
                    vlan-id 1112;
                    family inet {
                        address 10.100.1.1/31;
                    }
                    family iso;
                    family mpls;
                }
            }
            lo0 {
                unit 12 {
                    family inet {
                        address 10.100.0.12/32;
                    }
                    family iso {
                        address 49.0100.0101.0000.0012.00;
                    }
                }
            }
        }
        protocols {
            mpls {
                interface all;
            }
            isis {
                interface all;
            }
            ldp {
                interface all;
            }
        }
        routing-options {
            autonomous-system 100;
        }
    }
    13-ASBR {
        interfaces {
            ge-0/0/0 {
                unit 20 {
                    vlan-id 20;
                    family inet {
                        address 172.167.20.0/31;
                    }
                }
                unit 30 {
                    vlan-id 30;
                    family inet {
                        address 172.167.30.0/31;
                    }
                }
            }
            ge-0/0/1 {
                unit 1213 {
                    vlan-id 1213;
                    family inet {
                        address 10.100.1.3/31;
                    }
                    family iso;
                    family mpls;
                }
            }
            lo0 {
                unit 13 {
                    family inet {
                        address 10.100.0.13/32;
                    }
                    family iso {
                        address 49.0100.0101.0000.0013.00;
                    }
                }
            }
        }
        protocols {
            mpls {
                interface all;
            }
            bgp {
                group internal {
                    type internal;
                    local-address 10.100.0.13;
                    family inet-vpn {
                        any;
                    }
                    neighbor 10.100.0.11;
                }
            }
            isis {
                interface all;
            }
            ldp {
                interface all;
            }
        }
        routing-instances {
            Cust-20 {
                instance-type vrf;
                interface ge-0/0/0.20;
                route-distinguisher 10.100.0.13:20;
                vrf-target target:100:20;
                protocols {
                    bgp {
                        group to-AS200 {
                            type external;
                            peer-as 200;
                            neighbor 172.167.20.1;
                        }
                    }
                }
            }
            Cust-30 {
                instance-type vrf;
                interface ge-0/0/0.30;
                route-distinguisher 10.100.0.13:30;
                vrf-target target:100:30;
                protocols {
                    bgp {
                        group to-AS200 {
                            type external;
                            peer-as 200;
                            neighbor 172.167.30.1;
                        }
                    }
                }
            }
        }
        routing-options {
            autonomous-system 100;
        }
    }
    14-ASBR {
        interfaces {
            ge-0/0/0 {
                unit 1415 {
                    vlan-id 1415;
                    family inet {
                        address 10.200.1.4/31;
                    }
                    family iso;
                    family mpls;
                }
            }
            ge-0/0/1 {
                unit 20 {
                    vlan-id 20;
                    family inet {
                        address 172.167.20.1/31;
                    }
                }
                unit 30 {
                    vlan-id 30;
                    family inet {
                        address 172.167.30.1/31;
                    }
                }
            }
            lo0 {
                unit 14 {
                    family inet {
                        address 10.200.0.14/32;
                    }
                    family iso {
                        address 49.0200.0102.0000.0014.00;
                    }
                }
            }
        }
        protocols {
            mpls {
                interface all;
            }
            bgp {
                group internal {
                    type internal;
                    local-address 10.200.0.14;
                    family inet-vpn {
                        any;
                    }
                    neighbor 10.200.0.16;
                }
            }
            isis {
                interface all;
            }
            ldp {
                interface all;
            }
        }
        routing-instances {
            Cust-20 {
                instance-type vrf;
                interface ge-0/0/1.20;
                route-distinguisher 10.200.0.13:20;
                vrf-target target:200:20;
                protocols {
                    bgp {
                        group to-AS100 {
                            type external;
                            peer-as 100;
                            neighbor 172.167.20.0;
                        }
                    }
                }
            }
            Cust-30 {
                instance-type vrf;
                interface ge-0/0/1.30;
                route-distinguisher 10.200.0.14:30;
                vrf-target target:200:30;
                protocols {
                    bgp {
                        group to-AS100 {
                            type external;
                            peer-as 100;
                            neighbor 172.167.30.0;
                        }
                    }
                }
            }
        }
        routing-options {
            autonomous-system 200;
        }
    }
    15-P {
        interfaces {
            ge-0/0/0 {
                unit 1516 {
                    vlan-id 1516;
                    family inet {
                        address 10.200.1.6/31;
                    }
                    family iso;
                    family mpls;
                }
            }
            ge-0/0/1 {
                unit 1415 {
                    vlan-id 1415;
                    family inet {
                        address 10.200.1.5/31;
                    }
                    family iso;
                    family mpls;
                }
            }
            lo0 {
                unit 15 {
                    family inet {
                        address 10.200.0.15/32;
                    }
                    family iso {
                        address 49.0100.0102.0000.0015.00;
                    }
                }
            }
        }
        protocols {
            mpls {
                interface all;
            }
            isis {
                interface all;
            }
            ldp {
                interface all;
            }
        }
    }
    16-PE {
        interfaces {
            ge-0/0/0 {
                unit 1622 {
                    vlan-id 1622;
                    family inet {
                        address 192.168.20.2/31;
                    }
                }
                unit 1632 {
                    vlan-id 1632;
                    family inet {
                        address 192.168.30.2/31;
                    }
                }
            }
            ge-0/0/1 {
                unit 1516 {
                    vlan-id 1516;
                    family inet {
                        address 10.200.1.7/31;
                    }
                    family iso;
                    family mpls;
                }
            }
            lo0 {
                unit 16 {
                    family inet {
                        address 10.200.0.16/32;
                    }
                    family iso {
                        address 49.0200.0102.0000.0016.00;
                    }
                }
            }
        }
        protocols {
            mpls {
                interface all;
            }
            bgp {
                group internal {
                    type internal;
                    local-address 10.200.0.16;
                    family inet-vpn {
                        any;
                    }
                    neighbor 10.200.0.14;
                }
            }
            isis {
                interface all;
            }
            ldp {
                interface all;
            }
        }
        routing-instances {
            Cust-20 {
                instance-type vrf;
                interface ge-0/0/0.1622;
                route-distinguisher 10.100.0.16:20;
                vrf-target target:200:20;
                protocols {
                    bgp {
                        group to-Cust {
                            type external;
                            peer-as 20;
                            as-override;
                            neighbor 192.168.20.3;
                        }
                    }
                }
            }
            Cust-30 {
                instance-type vrf;
                interface ge-0/0/0.1632;
                route-distinguisher 10.100.0.16:30;
                vrf-target target:200:30;
                protocols {
                    bgp {
                        group to-Cust {
                            type external;
                            peer-as 30;
                            as-override;
                            neighbor 192.168.30.3;
                        }
                    }
                }
            }
        }
        routing-options {
            autonomous-system 200;
        }
    }
    21-CE {
        interfaces {
            ge-0/0/1 {
                unit 1121 {
                    vlan-id 1121;
                    family inet {
                        address 192.168.20.1/31;
                    }
                }
            }
            lo0 {
                unit 21 {
                    family inet {
                        address 192.168.0.21/32;
                    }
                }
            }
        }
        protocols {
            bgp {
                group AS100 {
                    type external;
                    export to-bgp;
                    peer-as 100;
                    neighbor 192.168.20.0;
                }
            }
        }
        policy-options {
            policy-statement to-bgp {
                from protocol direct;
                then accept;
            }
        }
        routing-options {
            autonomous-system 20;
        }
    }
    22-CE {
        interfaces {
            ge-0/0/1 {
                unit 1622 {
                    vlan-id 1622;
                    family inet {
                        address 192.168.20.3/31;
                    }
                }
            }
            lo0 {
                unit 22 {
                    family inet {
                        address 192.168.0.22/32;
                    }
                }
            }
        }
        protocols {
            bgp {
                group AS200 {
                    type external;
                    export to-bgp;
                    peer-as 200;
                    neighbor 192.168.20.2;
                }
            }
        }
        policy-options {
            policy-statement to-bgp {
                from protocol direct;
                then accept;
            }
        }
        routing-options {
            autonomous-system 20;
        }
    }
    31-CE {
        interfaces {
            ge-0/0/1 {
                unit 1131 {
                    vlan-id 1131;
                    family inet {
                        address 192.168.30.1/31;
                    }
                }
            }
            lo0 {
                unit 31 {
                    family inet {
                        address 192.168.0.31/32;
                    }
                }
            }
        }
        protocols {
            bgp {
                group AS100 {
                    type external;
                    export to-bgp;
                    peer-as 100;
                    neighbor 192.168.30.0;
                }
            }
        }
        policy-options {
            policy-statement to-bgp {
                from protocol direct;
                then accept;
            }
        }
        routing-options {
            autonomous-system 30;
        }
    }
    32-CE {
        interfaces {
            ge-0/0/1 {
                unit 1632 {
                    vlan-id 1632;
                    family inet {
                        address 192.168.30.3/31;
                    }
                }
            }
            lo0 {
                unit 32 {
                    family inet {
                        address 192.168.0.32/32;
                    }
                }
            }
        }
        protocols {
            bgp {
                group AS200 {
                    type external;
                    export to-bgp;
                    peer-as 200;
                    neighbor 192.168.30.2;
                }
            }
        }
        policy-options {
            policy-statement to-bgp {
                from protocol direct;
                then accept;
            }
        }
        routing-options {
            autonomous-system 30;
        }
    }
}
interfaces {
    ge-0/0/0 {
        description "to ge-0/0/1";
        vlan-tagging;
    }
    ge-0/0/1 {
        description "to ge-0/0/0";
        vlan-tagging;
    }
    ge-0/0/2 {
        description "to ge-0/0/3";
        vlan-tagging;
    }
    ge-0/0/3 {
        description "to ge-0/0/2";
        vlan-tagging;
    }
    ge-0/0/4 {
        description "to ge-0/0/5";
        vlan-tagging;
    }
    ge-0/0/5 {
        description "to ge-0/0/4";
        vlan-tagging;
    }
    ge-0/0/6 {
        description "to ge-0/0/7";
        vlan-tagging;
    }
    ge-0/0/7 {
        description "to ge-0/0/6";
        vlan-tagging;
    }
    ge-0/0/8 {
        description "to eth1";
    }
    ge-0/0/9 {
        description "to eth2";
    }
    fxp0 {
        unit 0 {
            family inet {
                address 192.168.83.11/24;
            }
        }
    }
}

Verification

[edit]
root@MX:21-CE# run show route 

inet.0: 6 destinations, 6 routes (6 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

172.167.20.0/31    *[BGP/170] 00:04:04, localpref 100
                      AS path: 100 I, validation-state: unverified
                    > to 192.168.20.0 via ge-0/0/1.1121
192.168.0.21/32    *[Direct/0] 04:10:00
                    > via lo0.21
192.168.0.22/32    *[BGP/170] 00:04:04, localpref 100
                      AS path: 100 200 100 I, validation-state: unverified
                    > to 192.168.20.0 via ge-0/0/1.1121
192.168.20.0/31    *[Direct/0] 04:09:08
                    > via ge-0/0/1.1121
192.168.20.1/32    *[Local/0] 04:09:08
                      Local via ge-0/0/1.1121
192.168.20.2/31    *[BGP/170] 00:04:04, localpref 100
                      AS path: 100 200 I, validation-state: unverified
                    > to 192.168.20.0 via ge-0/0/1.1121

[edit]
root@MX:21-CE# run ping 192.168.0.22 count 3 source 192.168.0.21 
PING 192.168.0.22 (192.168.0.22): 56 data bytes
64 bytes from 192.168.0.22: icmp_seq=0 ttl=52 time=9.114 ms
64 bytes from 192.168.0.22: icmp_seq=1 ttl=52 time=10.155 ms
64 bytes from 192.168.0.22: icmp_seq=2 ttl=52 time=8.088 ms

--- 192.168.0.22 ping statistics ---
3 packets transmitted, 3 packets received, 0% packet loss
round-trip min/avg/max/stddev = 8.088/9.119/10.155/0.844 ms

Label Operation


1. Customer router will send simple traffic.

root@MX:21-CE> show route 192.168.0.22 

inet.0: 6 destinations, 6 routes (6 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

192.168.0.22/32    *[BGP/170] 00:10:08, localpref 100
                      AS path: 100 200 100 I, validation-state: unverified
                    > to 192.168.20.0 via ge-0/0/1.1121

2. Router 11-PE will add two labels to the traffic, L3VPN label and LDP transport label.

root@MX:11-PE> show route 192.168.0.22 

Cust-20.inet.0: 6 destinations, 7 routes (6 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

192.168.0.22/32    *[BGP/170] 00:11:04, localpref 100, from 10.100.0.13
                      AS path: 200 20 I, validation-state: unverified
                    > to 10.100.1.1 via ge-0/0/0.1112, Push 299808, Push 299792(top)

3. Router 12-P is PHP and will pop the LDP label.

root@MX:12-P> show route 192.168.0.22 

root@MX:12-P> show route label 299792 

mpls.0: 8 destinations, 8 routes (8 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

299792             *[LDP/9] 02:34:46, metric 1
                    > to 10.100.1.3 via ge-0/0/0.1213, Pop      
299792(S=0)        *[LDP/9] 02:34:46, metric 1
                    > to 10.100.1.3 via ge-0/0/0.1213, Pop

4. Router 13-ASBR will pop L3VPN label and will forward simple traffic on the dedicated circuit.

root@MX:13-ASBR> show route 192.168.0.22 

Cust-20.inet.0: 6 destinations, 6 routes (6 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

192.168.0.22/32    *[BGP/170] 00:14:39, localpref 100
                      AS path: 200 20 I, validation-state

5. Router 14-ASBR will add two labels to the traffic, L3VPN label and LDP transport label.

root@MX:14-ASBR> show route 192.168.0.22 

Cust-20.inet.0: 6 destinations, 6 routes (6 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

192.168.0.22/32    *[BGP/170] 00:16:04, localpref 100, from 10.200.0.16
                      AS path: 20 I, validation-state: unverified
                    > to 10.200.1.5 via ge-0/0/0.1415, Push 299808, Push 299792(top)

6. Router 15-P is PHP and will pop the LDP label.

root@MX:15-P> show route 192.168.0.22 

root@MX:15-P> show route label 299792 

mpls.0: 8 destinations, 8 routes (8 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

299792             *[LDP/9] 00:23:26, metric 1
                    > to 10.200.1.7 via ge-0/0/0.1516, Pop      
299792(S=0)        *[LDP/9] 00:23:26, metric 1
                    > to 10.200.1.7 via ge-0/0/0.1516, Pop

7. Router 16-PE will pop L3VPN label and will forward simple traffic to the end customer.

root@MX:16-PE> show route 192.168.0.22 

Cust-20.inet.0: 6 destinations, 7 routes (6 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

192.168.0.22/32    *[BGP/170] 00:19:35, localpref 100
                      AS path: 20 I, validation-state: unverified
                    > to 192.168.20.3 via ge-0/0/0.1622

The Factors that Limit the Scalability of this Method

  • All inter-AS VPN routes (potentially a very large number) must be stored in the BGP RIBs and IP routing tables on the AS boundary routers.
  • You must configure VRFs on each AS boundary router.

Sources:

Carrier-of-Carriers with Internet Service Provider as the Customer on a vMX

Implementation description

In this article, I will build a Carrier-of-Carriers with Internet Service Provider as the customer setup on a single vMX. Each individual router will be configured as a logical router. The vMX router has a back-to-back connection between ge-0/0/0 and ge-0/0/1 ports. For each different link, a different VLAN number will be configured to separate the traffic generated based on router ID number.

Network Diagram

IP Allocation

  • AS 1: 192.168.0.0/16
    • CE 1: 192.168.0.1/32
    • CE 2: 192.168.0.2/32
    • Link 111: 192.168.1.0/31
    • Link 216: 192.168.1.2/31
  • AS 100: 10.10.0.0/16
    • PE 11: 10.10.0.11/32
    • P 12: 10.10.0.12/32
    • C-CE 13: 10.10.0.13/32
    • C-CE 14: 10.10.0.14/32
    • P 15: 10.10.0.15/32
    • PE 16: 10.10.0.16/32
    • Internal Links: 10.10.1.0/16
  • AS 200: 172.16.0.0/16
    • C-PE 21: 172.16.0.21/32
    • C-P 22: 172.16.0.22/32
    • C-PE 23: 172.16.0.23/32
    • Link 1321: 172.16.255.0/31
    • Link 1423: 172.16.255.2/31

Full Configuration

root@MX# show | no-more 
## Last changed: 2018-03-04 11:27:16 UTC
version 15.1F4.15;
groups {
    isis {
        logical-systems {
            <*> {
                protocols {
                    isis {
                        level 1 disable;
                        interface <*> {
                            point-to-point;
                        }
                    }
                }
            }
        }
    }
}
apply-groups isis;
system {
    host-name MX;
    root-authentication {
        encrypted-password "$5$L3F31155$kVyagZl2v/WM9s32/hi7VCXxM5o0vupYD.LO3uvCif4"; ## SECRET-DATA
    }
    services {
        ssh;
        netconf {
            ssh;
        }
    }
    syslog {
        user * {
            any emergency;
        }
        file messages {
            any notice;
            authorization info;
        }
        file interactive-commands {
            interactive-commands any;
        }
    }
}
logical-systems {
    1-CE {
        interfaces {
            ge-0/0/0 {
                unit 111 {
                    vlan-id 111;
                    family inet {
                        address 192.168.1.0/31;
                    }
                }
            }
            lo0 {
                unit 1 {
                    family inet {
                        address 192.168.0.1/32;
                    }
                }
            }
        }
        protocols {
            bgp {
                group AS100 {
                    type external;
                    export to-bgp;
                    peer-as 100;
                    neighbor 192.168.1.1;
                }
            }
        }
        policy-options {
            policy-statement to-bgp {
                from protocol direct;
                then accept;
            }
        }
        routing-options {
            autonomous-system 1;
        }
    }
    11-PE {
        interfaces {
            ge-0/0/0 {
                unit 1112 {
                    vlan-id 1112;
                    family inet {
                        address 10.10.1.0/31;
                    }
                    family iso;
                }
            }
            ge-0/0/1 {
                unit 111 {
                    vlan-id 111;
                    family inet {
                        address 192.168.1.1/31;
                    }
                }
            }
            lo0 {
                unit 11 {
                    family inet {
                        address 10.10.0.11/32;
                    }
                    family iso {
                        address 49.0100.0100.1000.0011.00;
                    }
                }
            }
        }
        protocols {
            bgp {
                group internal {
                    type internal;
                    local-address 10.10.0.11;
                    export to-ibgp;
                    neighbor 10.10.0.12;
                    neighbor 10.10.0.13;
                    neighbor 10.10.0.14;
                    neighbor 10.10.0.15;
                    neighbor 10.10.0.16;
                }
                group to-customer {
                    type external;
                    peer-as 1;
                    as-override;
                    neighbor 192.168.1.0;
                }
            }
            isis {
                interface ge-0/0/0.1112;
                interface lo0.11;
            }
        }
        policy-options {
            policy-statement to-ibgp {
                term nhs {
                    from protocol bgp;
                    then {
                        next-hop self;
                    }
                }
            }
        }
        routing-options {
            autonomous-system 100;
        }
    }
    12-P {
        interfaces {
            ge-0/0/0 {
                unit 1213 {
                    vlan-id 1213;
                    family inet {
                        address 10.10.1.2/31;
                    }
                    family iso;
                }
            }
            ge-0/0/1 {
                unit 1112 {
                    vlan-id 1112;
                    family inet {
                        address 10.10.1.1/31;
                    }
                    family iso;
                }
            }
            lo0 {
                unit 12 {
                    family inet {
                        address 10.10.0.12/32;
                    }
                    family iso {
                        address 49.0100.0100.1000.0012.00;
                    }
                }
            }
        }
        protocols {
            bgp {
                group internal {
                    type internal;
                    local-address 10.10.0.12;
                    neighbor 10.10.0.11;
                    neighbor 10.10.0.13;
                    neighbor 10.10.0.14;
                    neighbor 10.10.0.15;
                    neighbor 10.10.0.16;
                }
            }
            isis {
                interface all;
            }
        }
        routing-options {
            autonomous-system 100;
        }
    }
    13-C-CE {
        interfaces {
            ge-0/0/0 {
                unit 1321 {
                    vlan-id 1321;
                    family inet {
                        address 172.16.255.0/31;
                    }
                    family mpls;
                }
            }
            ge-0/0/1 {
                unit 1213 {
                    vlan-id 1213;
                    family inet {
                        address 10.10.1.3/31;
                    }
                    family iso;
                }
            }
            lo0 {
                unit 13 {
                    family inet {
                        address 10.10.0.13/32;
                    }
                    family iso {
                        address 49.0100.0100.1000.0013.00;
                    }
                }
            }
        }
        protocols {
            mpls {
                interface ge-0/0/0.1321;
            }
            bgp {
                group to-isp {
                    type external;
                    export internal;
                    peer-as 200;
                    neighbor 172.16.255.1 {
                        family inet {
                            labeled-unicast;
                        }
                    }
                }
                group internal {
                    type internal;
                    local-address 10.10.0.13;
                    export to-ibgp;
                    neighbor 10.10.0.11;
                    neighbor 10.10.0.12;
                    neighbor 10.10.0.14;
                    neighbor 10.10.0.15;
                    neighbor 10.10.0.16;
                }
            }
            isis {
                interface all;
            }
        }
        policy-options {
            policy-statement internal {
                term internal {
                    from protocol [ isis direct ];
                    then accept;
                }
                term reject {
                    then reject;
                }
            }
            policy-statement to-ibgp {
                term nhs {
                    from protocol bgp;
                    then {
                        next-hop self;
                    }
                }
            }
        }
        routing-options {
            autonomous-system 100;
        }
    }
    14-C-CE {
        interfaces {
            ge-0/0/0 {
                unit 1415 {
                    vlan-id 1415;
                    family inet {
                        address 10.10.1.4/31;
                    }
                    family iso;
                }
                unit 1423 {
                    vlan-id 1423;
                    family inet {
                        address 172.16.255.2/31;
                    }
                    family mpls;
                }
            }
            lo0 {
                unit 14 {
                    family inet {
                        address 10.10.0.14/32;
                    }
                    family iso {
                        address 49.0100.0100.1000.0014.00;
                    }
                }
            }
        }
        protocols {
            mpls {
                interface ge-0/0/0.1423;
            }
            bgp {
                group to-isp {
                    type external;
                    export internal;
                    peer-as 200;
                    neighbor 172.16.255.3 {
                        family inet {
                            labeled-unicast;
                        }
                    }
                }
                group internal {
                    type internal;
                    local-address 10.10.0.14;
                    export to-ibgp;
                    neighbor 10.10.0.11;
                    neighbor 10.10.0.12;
                    neighbor 10.10.0.13;
                    neighbor 10.10.0.15;
                    neighbor 10.10.0.16;
                }
            }
            isis {
                interface all;
            }
        }
        policy-options {
            policy-statement internal {
                term internal {
                    from protocol [ isis direct ];
                    then accept;
                }
                term reject {
                    then reject;
                }
            }
            policy-statement to-ibgp {
                term nhs {
                    from protocol bgp;
                    then {
                        next-hop self;
                    }
                }
            }
        }
        routing-options {
            autonomous-system 100;
        }
    }
    15-P {
        interfaces {
            ge-0/0/0 {
                unit 1516 {
                    vlan-id 1516;
                    family inet {
                        address 10.10.1.6/31;
                    }
                    family iso;
                }
            }
            ge-0/0/1 {
                unit 1415 {
                    vlan-id 1415;
                    family inet {
                        address 10.10.1.5/31;
                    }
                    family iso;
                }
            }
            lo0 {
                unit 15 {
                    family inet {
                        address 10.10.0.15/32;
                    }
                    family iso {
                        address 49.0100.0100.1000.0015.00;
                    }
                }
            }
        }
        protocols {
            bgp {
                group internal {
                    type internal;
                    local-address 10.10.0.15;
                    neighbor 10.10.0.11;
                    neighbor 10.10.0.12;
                    neighbor 10.10.0.13;
                    neighbor 10.10.0.14;
                    neighbor 10.10.0.16;
                }
            }
            isis {
                interface all;
            }
        }
        routing-options {
            autonomous-system 100;
        }
    }
    16-PE {
        interfaces {
            ge-0/0/1 {
                unit 216 {
                    vlan-id 216;
                    family inet {
                        address 192.168.1.3/31;
                    }
                }
                unit 1516 {
                    vlan-id 1516;
                    family inet {
                        address 10.10.1.7/31;
                    }
                    family iso;
                }
            }
            lo0 {
                unit 16 {
                    family inet {
                        address 10.10.0.16/32;
                    }
                    family iso {
                        address 49.0100.0100.1000.0016.00;
                    }
                }
            }
        }
        protocols {
            bgp {
                group internal {
                    type internal;
                    local-address 10.10.0.16;
                    export to-ibgp;
                    neighbor 10.10.0.11;
                    neighbor 10.10.0.12;
                    neighbor 10.10.0.13;
                    neighbor 10.10.0.14;
                    neighbor 10.10.0.15;
                }
                group to-customer {
                    type external;
                    peer-as 1;
                    as-override;
                    neighbor 192.168.1.2;
                }
            }
            isis {
                interface ge-0/0/1.1516;
                interface lo0.16;
            }
        }
        policy-options {
            policy-statement to-ibgp {
                term nhs {
                    from protocol bgp;
                    then {
                        next-hop self;
                    }
                }
            }
        }
        routing-options {
            autonomous-system 100;
        }
    }
    2-CE {
        interfaces {
            ge-0/0/0 {
                unit 216 {
                    vlan-id 216;
                    family inet {
                        address 192.168.1.2/31;
                    }
                }
            }
            lo0 {
                unit 2 {
                    family inet {
                        address 192.168.0.2/32;
                    }
                }
            }
        }
        protocols {
            bgp {
                group AS100 {
                    type external;
                    export to-bgp;
                    peer-as 100;
                    neighbor 192.168.1.3;
                }
            }
        }
        policy-options {
            policy-statement to-bgp {
                from protocol direct;
                then accept;
            }
        }
        routing-options {
            autonomous-system 1;
        }
    }
    21-C-PE {
        interfaces {
            ge-0/0/0 {
                unit 2122 {
                    vlan-id 2122;
                    family inet {
                        address 172.16.1.0/31;
                    }
                    family iso;
                    family mpls;
                }
            }
            ge-0/0/1 {
                unit 1321 {
                    vlan-id 1321;
                    family inet {
                        address 172.16.255.1/31;
                    }
                    family mpls;
                }
            }
            lo0 {
                unit 21 {
                    family inet {
                        address 172.16.0.21/32;
                    }
                    family iso {
                        address 49.0200.1720.1600.0021.00;
                    }
                }
            }
        }
        protocols {
            mpls {
                interface all;
            }
            bgp {
                group pe-pe {
                    type internal;
                    local-address 172.16.0.21;
                    neighbor 172.16.0.23 {
                        family inet-vpn {
                            any;
                        }
                    }
                }
            }
            isis {
                interface all;
            }
            ldp {
                interface ge-0/0/0.2122;
                interface lo0.21;
            }
        }
        routing-instances {
            vpn-isp {
                instance-type vrf;
                interface ge-0/0/1.1321;
                route-distinguisher 172.16.0.21:100;
                vrf-target target:200:100;
                protocols {
                    bgp {
                        group to-isp {
                            peer-as 100;
                            neighbor 172.16.255.0 {
                                family inet {
                                    labeled-unicast;
                                }
                                as-override;
                            }
                        }
                    }
                    mpls {
                        interface all;
                    }
                }
            }
        }
        routing-options {
            autonomous-system 200;
        }
    }
    22-C-P {
        interfaces {
            ge-0/0/0 {
                unit 2223 {
                    vlan-id 2223;
                    family inet {
                        address 172.16.1.2/31;
                    }
                    family iso;
                    family mpls;
                }
            }
            ge-0/0/1 {
                unit 2122 {
                    vlan-id 2122;
                    family inet {
                        address 172.16.1.1/31;
                    }
                    family iso;
                    family mpls;
                }
            }
            lo0 {
                unit 22 {
                    family inet {
                        address 172.16.0.22/32;
                    }
                    family iso {
                        address 49.0200.1720.1600.0022.00;
                    }
                }
            }
        }
        protocols {
            mpls {
                interface all;
            }
            isis {
                interface all;
            }
            ldp {
                interface ge-0/0/0.2223;
                interface ge-0/0/1.2122;
                interface lo0.22;
            }
        }
    }
    23-C-PE {
        interfaces {
            ge-0/0/1 {
                unit 1423 {
                    vlan-id 1423;
                    family inet {
                        address 172.16.255.3/31;
                    }
                    family mpls;
                }
                unit 2223 {
                    vlan-id 2223;
                    family inet {
                        address 172.16.1.3/31;
                    }
                    family iso;
                    family mpls;
                }
            }
            lo0 {
                unit 23 {
                    family inet {
                        address 172.16.0.23/32;
                    }
                    family iso {
                        address 49.0200.1720.1600.0023.00;
                    }
                }
            }
        }
        protocols {
            mpls {
                interface all;
            }
            bgp {
                group pe-pe {
                    type internal;
                    local-address 172.16.0.23;
                    neighbor 172.16.0.21 {
                        family inet-vpn {
                            any;
                        }
                    }
                }
            }
            isis {
                interface all;
            }
            ldp {
                interface ge-0/0/1.2223;
                interface lo0.23;
            }
        }
        routing-instances {
            vpn-isp {
                instance-type vrf;
                interface ge-0/0/1.1423;
                route-distinguisher 172.16.0.23:100;
                vrf-target target:200:100;
                protocols {
                    bgp {
                        group to-isp {
                            peer-as 100;
                            neighbor 172.16.255.2 {
                                family inet {
                                    labeled-unicast;
                                }
                                as-override;
                            }
                        }
                    }
                    mpls {
                        interface all;
                    }
                }
            }
        }
        routing-options {
            autonomous-system 200;
        }
    }
}
interfaces {
    ge-0/0/0 {
        description "to ge-0/0/1";
        vlan-tagging;
    }
    ge-0/0/1 {
        description "to ge-0/0/0";
        vlan-tagging;
    }
    ge-0/0/2 {
        description "to ge-0/0/3";
        vlan-tagging;
    }
    ge-0/0/3 {
        description "to ge-0/0/2";
        vlan-tagging;
    }
    ge-0/0/4 {
        description "to ge-0/0/5";
        vlan-tagging;
    }
    ge-0/0/5 {
        description "to ge-0/0/4";
        vlan-tagging;
    }
    ge-0/0/6 {
        description "to ge-0/0/7";
        vlan-tagging;
    }
    ge-0/0/7 {
        description "to ge-0/0/6";
        vlan-tagging;
    }
    ge-0/0/8 {
        description "to eth1";
    }
    ge-0/0/9 {
        description "to eth2";
    }
    fxp0 {
        unit 0 {
            family inet {
                address 192.168.83.11/24;
            }
        }
    }
}

Verification

root@MX:1-CE> show route                             

inet.0: 9 destinations, 9 routes (9 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

10.10.0.14/32      *[BGP/170] 00:17:04, localpref 100
                      AS path: 100 200 200 I, validation-state: unverified
                    > to 192.168.1.1 via ge-0/0/0.111
10.10.0.15/32      *[BGP/170] 00:17:04, localpref 100
                      AS path: 100 200 200 I, validation-state: unverified
                    > to 192.168.1.1 via ge-0/0/0.111
10.10.0.16/32      *[BGP/170] 00:17:04, localpref 100
                      AS path: 100 200 200 I, validation-state: unverified
                    > to 192.168.1.1 via ge-0/0/0.111
10.10.1.4/31       *[BGP/170] 00:17:04, localpref 100
                      AS path: 100 200 200 I, validation-state: unverified
                    > to 192.168.1.1 via ge-0/0/0.111
10.10.1.6/31       *[BGP/170] 00:17:04, localpref 100
                      AS path: 100 200 200 I, validation-state: unverified
                    > to 192.168.1.1 via ge-0/0/0.111
192.168.0.1/32     *[Direct/0] 03:17:20
                    > via lo0.1
192.168.0.2/32     *[BGP/170] 00:08:30, localpref 100
                      AS path: 100 100 I, validation-state: unverified
                    > to 192.168.1.1 via ge-0/0/0.111
192.168.1.0/31     *[Direct/0] 03:16:25
                    > via ge-0/0/0.111
192.168.1.0/32     *[Local/0] 03:16:25
                      Local via ge-0/0/0.111

root@MX:1-CE> ping 192.168.0.2 source 192.168.0.1 count 3 
PING 192.168.0.2 (192.168.0.2): 56 data bytes
64 bytes from 192.168.0.2: icmp_seq=0 ttl=46 time=12.347 ms
64 bytes from 192.168.0.2: icmp_seq=1 ttl=46 time=10.568 ms
64 bytes from 192.168.0.2: icmp_seq=2 ttl=46 time=7.953 ms

--- 192.168.0.2 ping statistics ---
3 packets transmitted, 3 packets received, 0% packet loss
round-trip min/avg/max/stddev = 7.953/10.289/12.347/1.805 ms

Label Operation

1. From router 1-CE to 13-C-CE will flow simple traffic.

root@MX:11-PE> show route 192.168.0.2 

inet.0: 15 destinations, 21 routes (15 active, 0 holddown, 1 hidden)
+ = Active Route, - = Last Active, * = Both

192.168.0.2/32     *[BGP/170] 00:10:57, localpref 100, from 10.10.0.16
                      AS path: 1 I, validation-state: unverified
                    > to 10.10.1.1 via ge-0/0/0.1112

2. Router 13-C-CE will push BGP-LU label.

root@MX:13-C-CE> show route 192.168.0.2 

inet.0: 15 destinations, 20 routes (15 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

192.168.0.2/32     *[BGP/170] 00:11:33, localpref 100, from 10.10.0.16
                      AS path: 1 I, validation-state: unverified
                    > to 172.16.255.1 via ge-0/0/0.1321, Push 299952

3. Router 21-C-PE will swap BGP-LU label and will push an LDP label.

root@MX:21-C-PE> show route 192.168.0.2 

root@MX:21-C-PE> show route label 299952 

vpn-isp.mpls.0: 6 destinations, 6 routes (6 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

299952             *[VPN/170] 00:35:44, metric2 1, from 172.16.0.23
                    > to 172.16.1.1 via ge-0/0/0.2122, Swap 299952, Push 299792(top)

root@MX:21-C-PE> show route label 299792    

mpls.0: 10 destinations, 10 routes (10 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

299792             *[LDP/9] 03:22:12, metric 1
                    > to 172.16.1.1 via ge-0/0/0.2122, Swap 299792

root@MX:21-C-PE> show route 172.16.0.23/32 protocol ldp 

inet.0: 7 destinations, 7 routes (7 active, 0 holddown, 0 hidden)

inet.3: 2 destinations, 2 routes (2 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

172.16.0.23/32     *[LDP/9] 03:24:05, metric 1
                    > to 172.16.1.1 via ge-0/0/0.2122, Push 299792

4. Router 22-C-P is PHP router and will pop the LDP label.

root@MX:22-C-P> show route label 299792 

mpls.0: 8 destinations, 8 routes (8 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

299792             *[LDP/9] 03:24:49, metric 1
                    > to 172.16.1.3 via ge-0/0/0.2223, Pop      
299792(S=0)        *[LDP/9] 03:24:49, metric 1
                    > to 172.16.1.3 via ge-0/0/0.2223, Pop   

5. Router 23-C-PE will swap BGP-LU label.

root@MX:23-C-PE> show route label 299952  

mpls.0: 10 destinations, 10 routes (10 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

299952             *[VPN/170] 00:42:46
                    > to 172.16.255.2 via ge-0/0/1.1423, Swap 299936

6. Router 14-C-CE will pop the BGP-LU label and will forward simple traffic.

root@MX:14-C-CE> show route label 299936 

mpls.0: 6 destinations, 6 routes (6 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

299936             *[VPN/170] 00:44:10
                    > to 10.10.1.5 via ge-0/0/0.1415, Pop      
299936(S=0)        *[VPN/170] 00:44:10
                    > to 10.10.1.5 via ge-0/0/0.1415, Pop      

root@MX:14-C-CE> show route 192.168.0.2 

inet.0: 15 destinations, 20 routes (15 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

192.168.0.2/32     *[BGP/170] 00:21:54, localpref 100, from 10.10.0.16
                      AS path: 1 I, validation-state: unverified
                    > to 10.10.1.5 via ge-0/0/0.1415

Documentation

Carrier-of-Carriers with VPN Service Provider as the Customer on a vMX

Implementation description

In this article, I will build a Carrier-of-Carriers with VPN Service Provider as the customer setup on a single vMX. Each individual router will be configured as a logical router. The vMX router has a back-to-back connection between ge-0/0/0 and ge-0/0/1 ports. For each different link, a different VLAN number will be configured to separate the traffic generated based on router ID number.

Network Diagram

IP Allocation

  • AS 1: 192.168.0.0/16
    • CE 1: 192.168.0.1/32
    • CE 2: 192.168.0.2/32
    • Link 111: 192.168.1.0/31
    • Link 216: 192.168.1.2/31
  • AS 100: 10.10.0.0/16
    • PE 11: 10.10.0.11/32
    • P 12: 10.10.0.12/32
    • C-CE 13: 10.10.0.13/32
    • C-CE 14: 10.10.0.14/32
    • P 15: 10.10.0.15/32
    • PE 16: 10.10.0.16/32
    • Internal Links: 10.10.1.0/16
  • AS 200: 172.16.0.0/16
    • C-PE 21: 172.16.0.21/32
    • C-P 22: 172.16.0.22/32
    • C-PE 23: 172.16.0.23/32
    • Link 1321: 172.16.255.0/31
    • Link 1423: 172.16.255.2/31

Full Configuration

root@MX# show | no-more 
## Last changed: 2017-12-27 03:07:53 UTC
version 15.1F4.15;
groups {
    isis {
        logical-systems {
            <*> {
                protocols {
                    isis {
                        level 1 disable;
                        interface <*> {
                            point-to-point;
                        }
                    }
                }
            }
        }
    }
}
apply-groups isis;
system {
    host-name MX;
    root-authentication {
        encrypted-password "$5$L3F31155$kVyagZl2v/WM9s32/hi7VCXxM5o0vupYD.LO3uvCif4"; ## SECRET-DATA
    }
    services {
        ssh;
        netconf {
            ssh;
        }
    }
    syslog {
        user * {
            any emergency;
        }
        file messages {
            any notice;
            authorization info;
        }
        file interactive-commands {
            interactive-commands any;
        }
    }
}
logical-systems {
    1-CE {
        interfaces {
            ge-0/0/0 {
                unit 111 {
                    vlan-id 111;
                    family inet {
                        address 192.168.1.0/31;
                    }
                }
            }
            lo0 {
                unit 1 {
                    family inet {
                        address 192.168.0.1/32;
                    }
                }
            }
        }
        protocols {
            bgp {
                group AS100 {
                    type external;
                    export to-bgp;
                    peer-as 100;
                    neighbor 192.168.1.1;
                }
            }
        }
        policy-options {
            policy-statement to-bgp {
                from protocol direct;
                then accept;
            }
        }
        routing-options {
            autonomous-system 1;
        }
    }
    11-PE {
        interfaces {
            ge-0/0/0 {
                unit 1112 {
                    vlan-id 1112;
                    family inet {
                        address 10.10.1.0/31;
                    }
                    family iso;
                    family mpls;
                }
            }
            ge-0/0/1 {
                unit 111 {
                    vlan-id 111;
                    family inet {
                        address 192.168.1.1/31;
                    }
                }
            }
            lo0 {
                unit 11 {
                    family inet {
                        address 10.10.0.11/32;
                    }
                    family iso {
                        address 49.0100.0100.1000.0011.00;
                    }
                }
            }
        }
        protocols {
            mpls {
                interface all;
            }
            bgp {
                group internal {
                    type internal;
                    local-address 10.10.0.11;
                    neighbor 10.10.0.13 {
                        family inet {
                            labeled-unicast {
                                resolve-vpn;
                            }
                        }
                    }
                    neighbor 10.10.0.16 {
                        family inet-vpn {
                            any;
                        }
                    }
                }
            }
            isis {
                interface ge-0/0/0.1112;
                interface lo0.11;
            }
            ldp {
                interface ge-0/0/0.1112;
                interface lo0.11;
            }
        }
        routing-instances {
            VPN {
                instance-type vrf;
                interface ge-0/0/1.111;
                route-distinguisher 10.10.0.11:1;
                vrf-target target:100:1;
                vrf-table-label;
                protocols {
                    bgp {
                        group vpn {
                            peer-as 1;
                            as-override;
                            neighbor 192.168.1.0;
                        }
                    }
                }
            }
        }
        routing-options {
            autonomous-system 100;
        }
    }
    12-P {
        interfaces {
            ge-0/0/0 {
                unit 1213 {
                    vlan-id 1213;
                    family inet {
                        address 10.10.1.2/31;
                    }
                    family iso;
                    family mpls;
                }
            }
            ge-0/0/1 {
                unit 1112 {
                    vlan-id 1112;
                    family inet {
                        address 10.10.1.1/31;
                    }
                    family iso;
                    family mpls;
                }
            }
            lo0 {
                unit 12 {
                    family inet {
                        address 10.10.0.12/32;
                    }
                    family iso {
                        address 49.0100.0100.1000.0012.00;
                    }
                }
            }
        }
        protocols {
            mpls {
                traffic-engineering {
                    bgp-igp;
                }
                interface all;
            }
            isis {
                interface all;
            }
            ldp {
                interface ge-0/0/0.1213;
                interface ge-0/0/1.1112;
                interface lo0.12;
            }
        }
    }
    13-C-CE {
        interfaces {
            ge-0/0/0 {
                unit 1321 {
                    vlan-id 1321;
                    family inet {
                        address 172.16.255.0/31;
                    }
                    family mpls;
                }
            }
            ge-0/0/1 {
                unit 1213 {
                    vlan-id 1213;
                    family inet {
                        address 10.10.1.3/31;
                    }
                    family iso;
                    family mpls;
                }
            }
            lo0 {
                unit 13 {
                    family inet {
                        address 10.10.0.13/32;
                    }
                    family iso {
                        address 49.0100.0100.1000.0013.00;
                    }
                }
            }
        }
        protocols {
            mpls {
                traffic-engineering {
                    bgp-igp;
                }
                interface all;
            }
            bgp {
                group internal {
                    type internal;
                    local-address 10.10.0.13;
                    neighbor 10.10.0.11 {
                        family inet {
                            labeled-unicast;
                        }
                    }
                }
                group to-isp {
                    type external;
                    export internal;
                    peer-as 200;
                    neighbor 172.16.255.1 {
                        family inet {
                            labeled-unicast;
                        }
                    }
                }
            }
            isis {
                interface all;
            }
            ldp {
                interface ge-0/0/1.1213;
                interface lo0.13;
            }
        }
        policy-options {
            policy-statement internal {
                term internal {
                    from protocol [ isis direct ldp ];
                    then accept;
                }
                term reject {
                    then reject;
                }
            }
        }
        routing-options {
            autonomous-system 100;
        }
    }
    14-C-CE {
        interfaces {
            ge-0/0/0 {
                unit 1415 {
                    vlan-id 1415;
                    family inet {
                        address 10.10.1.4/31;
                    }
                    family iso;
                    family mpls;
                }
                unit 1423 {
                    vlan-id 1423;
                    family inet {
                        address 172.16.255.2/31;
                    }
                    family mpls;
                }
            }
            lo0 {
                unit 14 {
                    family inet {
                        address 10.10.0.14/32;
                    }
                    family iso {
                        address 49.0100.0100.1000.0014.00;
                    }
                }
            }
        }
        protocols {
            mpls {
                traffic-engineering {
                    bgp-igp;
                }
                interface all;
            }
            bgp {
                group internal {
                    type internal;
                    local-address 10.10.0.14;
                    neighbor 10.10.0.16 {
                        family inet {
                            labeled-unicast;
                        }
                    }
                }
                group to-isp {
                    type external;
                    export internal;
                    peer-as 200;
                    neighbor 172.16.255.3 {
                        family inet {
                            labeled-unicast;
                        }
                    }
                }
            }
            isis {
                interface all;
            }
            ldp {
                interface ge-0/0/0.1415;
                interface lo0.14;
            }
        }
        policy-options {
            policy-statement internal {
                term internal {
                    from protocol [ isis direct ldp ];
                    then accept;
                }
                term reject {
                    then reject;
                }
            }
        }
        routing-options {
            autonomous-system 100;
        }
    }
    15-P {
        interfaces {
            ge-0/0/0 {
                unit 1516 {
                    vlan-id 1516;
                    family inet {
                        address 10.10.1.6/31;
                    }
                    family iso;
                    family mpls;
                }
            }
            ge-0/0/1 {
                unit 1415 {
                    vlan-id 1415;
                    family inet {
                        address 10.10.1.5/31;
                    }
                    family iso;
                    family mpls;
                }
            }
            lo0 {
                unit 15 {
                    family inet {
                        address 10.10.0.15/32;
                    }
                    family iso {
                        address 49.0100.0100.1000.0015.00;
                    }
                }
            }
        }
        protocols {
            mpls {
                traffic-engineering {
                    bgp-igp;
                }
                interface all;
            }
            isis {
                interface all;
            }
            ldp {
                interface ge-0/0/0.1516;
                interface ge-0/0/1.1415;
                interface lo0.15;
            }
        }
    }
    16-PE {
        interfaces {
            ge-0/0/1 {
                unit 216 {
                    vlan-id 216;
                    family inet {
                        address 192.168.1.3/31;
                    }
                }
                unit 1516 {
                    vlan-id 1516;
                    family inet {
                        address 10.10.1.7/31;
                    }
                    family iso;
                    family mpls;
                }
            }
            lo0 {
                unit 16 {
                    family inet {
                        address 10.10.0.16/32;
                    }
                    family iso {
                        address 49.0100.0100.1000.0016.00;
                    }
                }
            }
        }
        protocols {
            mpls {
                interface all;
            }
            bgp {
                group internal {
                    type internal;
                    local-address 10.10.0.16;
                    neighbor 10.10.0.11 {
                        family inet-vpn {
                            any;
                        }
                    }
                    neighbor 10.10.0.14 {
                        family inet {
                            labeled-unicast {
                                resolve-vpn;
                            }
                        }
                    }
                }
            }
            isis {
                interface ge-0/0/1.1516;
                interface lo0.16;
            }
            ldp {
                interface ge-0/0/1.1516;
                interface lo0.16;
            }
        }
        routing-instances {
            VPN {
                instance-type vrf;
                interface ge-0/0/1.216;
                route-distinguisher 10.10.0.16:1;
                vrf-target target:100:1;
                vrf-table-label;
                protocols {
                    bgp {
                        group vpn {
                            peer-as 1;
                            as-override;
                            neighbor 192.168.1.2;
                        }
                    }
                }
            }
        }
        routing-options {
            autonomous-system 100;
        }
    }
    2-CE {
        interfaces {
            ge-0/0/0 {
                unit 216 {
                    vlan-id 216;
                    family inet {
                        address 192.168.1.2/31;
                    }
                }
            }
            lo0 {
                unit 2 {
                    family inet {
                        address 192.168.0.2/32;
                    }
                }
            }
        }
        protocols {
            bgp {
                group AS100 {
                    type external;
                    export to-bgp;
                    peer-as 100;
                    neighbor 192.168.1.3;
                }
            }
        }
        policy-options {
            policy-statement to-bgp {
                from protocol direct;
                then accept;
            }
        }
        routing-options {
            autonomous-system 1;
        }
    }
    21-C-PE {
        interfaces {
            ge-0/0/0 {
                unit 2122 {
                    vlan-id 2122;
                    family inet {
                        address 172.16.1.0/31;
                    }
                    family iso;
                    family mpls;
                }
            }
            ge-0/0/1 {
                unit 1321 {
                    vlan-id 1321;
                    family inet {
                        address 172.16.255.1/31;
                    }
                    family mpls;
                }
            }
            lo0 {
                unit 21 {
                    family inet {
                        address 172.16.0.21/32;
                    }
                    family iso {
                        address 49.0200.1720.1600.0021.00;
                    }
                }
            }
        }
        protocols {
            mpls {
                interface all;
            }
            bgp {
                group pe-pe {
                    type internal;
                    local-address 172.16.0.21;
                    neighbor 172.16.0.23 {
                        family inet-vpn {
                            any;
                        }
                    }
                }
            }
            isis {
                interface all;
            }
            ldp {
                interface ge-0/0/0.2122;
                interface lo0.21;
            }
        }
        routing-instances {
            vpn-isp {
                instance-type vrf;
                interface ge-0/0/1.1321;
                route-distinguisher 172.16.0.21:100;
                vrf-target target:200:100;
                protocols {
                    bgp {
                        group to-isp {
                            peer-as 100;
                            neighbor 172.16.255.0 {
                                family inet {
                                    labeled-unicast;
                                }
                                as-override;
                            }
                        }
                    }
                    mpls {
                        interface all;
                    }
                }
            }
        }
        routing-options {
            autonomous-system 200;
        }
    }
    22-C-P {
        interfaces {
            ge-0/0/0 {
                unit 2223 {
                    vlan-id 2223;
                    family inet {
                        address 172.16.1.2/31;
                    }
                    family iso;
                    family mpls;
                }
            }
            ge-0/0/1 {
                unit 2122 {
                    vlan-id 2122;
                    family inet {
                        address 172.16.1.1/31;
                    }
                    family iso;
                    family mpls;
                }
            }
            lo0 {
                unit 22 {
                    family inet {
                        address 172.16.0.22/32;
                    }
                    family iso {
                        address 49.0200.1720.1600.0022.00;
                    }
                }
            }
        }
        protocols {
            mpls {
                interface all;
            }
            isis {
                interface all;
            }
            ldp {
                interface ge-0/0/0.2223;
                interface ge-0/0/1.2122;
                interface lo0.22;
            }
        }
    }
    23-C-PE {
        interfaces {
            ge-0/0/1 {
                unit 1423 {
                    vlan-id 1423;
                    family inet {
                        address 172.16.255.3/31;
                    }
                    family mpls;
                }
                unit 2223 {
                    vlan-id 2223;
                    family inet {
                        address 172.16.1.3/31;
                    }
                    family iso;
                    family mpls;
                }
            }
            lo0 {
                unit 23 {
                    family inet {
                        address 172.16.0.23/32;
                    }
                    family iso {
                        address 49.0200.1720.1600.0023.00;
                    }
                }
            }
        }
        protocols {
            mpls {
                interface all;
            }
            bgp {
                group pe-pe {
                    type internal;
                    local-address 172.16.0.23;
                    neighbor 172.16.0.21 {
                        family inet-vpn {
                            any;
                        }
                    }
                }
            }
            isis {
                interface all;
            }
            ldp {
                interface ge-0/0/1.2223;
                interface lo0.23;
            }
        }
        routing-instances {
            vpn-isp {
                instance-type vrf;
                interface ge-0/0/1.1423;
                route-distinguisher 172.16.0.23:100;
                vrf-target target:200:100;
                protocols {
                    bgp {
                        group to-isp {
                            peer-as 100;
                            neighbor 172.16.255.2 {
                                family inet {
                                    labeled-unicast;
                                }
                                as-override;
                            }
                        }
                    }
                    mpls {
                        interface all;
                    }
                }
            }
        }
        routing-options {
            autonomous-system 200;
        }
    }
}
interfaces {
    ge-0/0/0 {
        description "to ge-0/0/1";
        vlan-tagging;
    }
    ge-0/0/1 {
        description "to ge-0/0/0";
        vlan-tagging;
    }
    ge-0/0/2 {
        description "to ge-0/0/3";
        vlan-tagging;
    }
    ge-0/0/3 {
        description "to ge-0/0/2";
        vlan-tagging;
    }
    ge-0/0/4 {
        description "to ge-0/0/5";
        vlan-tagging;
    }
    ge-0/0/5 {
        description "to ge-0/0/4";
        vlan-tagging;
    }
    ge-0/0/6 {
        description "to ge-0/0/7";
        vlan-tagging;
    }
    ge-0/0/7 {
        description "to ge-0/0/6";
        vlan-tagging;
    }
    ge-0/0/8 {
        description "to eth1";
    }
    ge-0/0/9 {
        description "to eth2";
    }
    fxp0 {
        unit 0 {
            family inet {
                address 192.168.83.11/24;
            }
        }
    }
}

Verification

root@MX:1-CE> show route 

inet.0: 5 destinations, 5 routes (5 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

192.168.0.1/32     *[Direct/0] 07:13:24
                    > via lo0.1
192.168.0.2/32     *[BGP/170] 00:05:32, localpref 100
                      AS path: 100 100 I, validation-state: unverified
                    > to 192.168.1.1 via ge-0/0/0.111
192.168.1.0/31     *[Direct/0] 01:40:58
                    > via ge-0/0/0.111
192.168.1.0/32     *[Local/0] 01:40:59
                      Local via ge-0/0/0.111
192.168.1.2/31     *[BGP/170] 00:05:32, localpref 100
                      AS path: 100 I, validation-state: unverified
                    > to 192.168.1.1 via ge-0/0/0.111

root@MX:1-CE> ping 192.168.0.2 source 192.168.0.1 count 3 
PING 192.168.0.2 (192.168.0.2): 56 data bytes
64 bytes from 192.168.0.2: icmp_seq=0 ttl=46 time=21.322 ms
64 bytes from 192.168.0.2: icmp_seq=1 ttl=46 time=13.263 ms
64 bytes from 192.168.0.2: icmp_seq=2 ttl=46 time=14.254 ms

--- 192.168.0.2 ping statistics ---
3 packets transmitted, 3 packets received, 0% packet loss
round-trip min/avg/max/stddev = 13.263/16.280/21.322/3.588 ms

Label Operation

1. Router 1-CE will send simple traffic:

root@MX:1-CE> show route 192.168.0.2  

inet.0: 5 destinations, 5 routes (5 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

192.168.0.2/32     *[BGP/170] 00:29:05, localpref 100
                      AS path: 100 100 I, validation-state: unverified
                    > to 192.168.1.1 via ge-0/0/0.111

2. Router 11-PE will push L3VPN label, BGP-LU label and LDP trasport label:

root@MX:11-PE> show route 192.168.0.2 

VPN.inet.0: 5 destinations, 6 routes (5 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

192.168.0.2/32     *[BGP/170] 00:30:40, localpref 100, from 10.10.0.16
                      AS path: 1 I, validation-state: unverified
                    > to 10.10.1.1 via ge-0/0/0.1112, Push 16, Push 300112, Push 299856(top)

3. Router 12-P is PHP and will pop LDP label:

root@MX:12-P> show route table mpls.0 label 299856 

mpls.0: 8 destinations, 8 routes (8 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

299856             *[LDP/9] 01:22:43, metric 1
                    > to 10.10.1.3 via ge-0/0/0.1213, Pop      
299856(S=0)        *[LDP/9] 01:22:43, metric 1
                    > to 10.10.1.3 via ge-0/0/0.1213, Pop      

3. Router 13-C-CE is transit router for BGP-LU tunnel and will swap BGP label:

root@MX:13-C-CE> show route label 300112 

mpls.0: 16 destinations, 16 routes (16 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

300112             *[VPN/170] 00:36:20
                    > to 172.16.255.1 via ge-0/0/0.1321, Swap 300096

4. Router 21-C-PE will swap BGP label and push LDP label:

root@MX:21-C-PE> show route label 300096  

vpn-isp.mpls.0: 8 destinations, 8 routes (8 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

300096             *[VPN/170] 00:38:24, metric2 1, from 172.16.0.23
                    > to 172.16.1.1 via ge-0/0/0.2122, Swap 300080, Push 299840(top)

root@MX:21-C-PE> show route 192.168.0.2 

root@MX:21-C-PE> show route 10.10.0.16 

vpn-isp.inet.0: 12 destinations, 13 routes (12 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

10.10.0.16/32      *[BGP/170] 00:39:31, MED 1, localpref 100, from 172.16.0.23
                      AS path: 100 I, validation-state: unverified
                    > to 172.16.1.1 via ge-0/0/0.2122, Push 300080, Push 299840(top)

root@MX:21-C-PE> show route 172.16.0.23 

inet.0: 7 destinations, 7 routes (7 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

172.16.0.23/32     *[IS-IS/18] 02:17:08, metric 20
                    > to 172.16.1.1 via ge-0/0/0.2122

inet.3: 2 destinations, 2 routes (2 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

172.16.0.23/32     *[LDP/9] 02:17:08, metric 1
                    > to 172.16.1.1 via ge-0/0/0.2122, Push 299840

4. Router 22-C-P is PHP and will pop LDP label:

root@MX:22-C-P> show route label 299840 

mpls.0: 8 destinations, 8 routes (8 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

299840             *[LDP/9] 02:19:45, metric 1
                    > to 172.16.1.3 via ge-0/0/0.2223, Pop      
299840(S=0)        *[LDP/9] 02:19:45, metric 1
                    > to 172.16.1.3 via ge-0/0/0.2223, Pop      

5. Router 23-C-PE will swap BGP label:

root@MX:23-C-PE> show route 10.10.0.16 

vpn-isp.inet.0: 12 destinations, 13 routes (12 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

10.10.0.16/32      *[BGP/170] 00:45:36, MED 1, localpref 100
                      AS path: 100 I, validation-state: unverified
                    > to 172.16.255.2 via ge-0/0/1.1423, Push 300064

root@MX:23-C-PE> show route table mpls.0 label 300080 

mpls.0: 12 destinations, 12 routes (12 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

300080             *[VPN/170] 00:56:09
                    > to 172.16.255.2 via ge-0/0/1.1423, Swap 300064

6. Router 14-C-CE will swap LDP label:

root@MX:14-C-CE> show route label 300064 

mpls.0: 16 destinations, 16 routes (16 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

300064             *[VPN/170] 00:57:14
                    > to 10.10.1.5 via ge-0/0/0.1415, Swap 299872

root@MX:14-C-CE> show route 10.10.0.16 

inet.0: 14 destinations, 16 routes (14 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

10.10.0.16/32      *[LDP/9] 01:47:03, metric 1
                    > to 10.10.1.5 via ge-0/0/0.1415, Push 299872
                    [IS-IS/18] 01:47:03, metric 20
                    > to 10.10.1.5 via ge-0/0/0.1415

Note: Here BGP-LU is, in fact, internal LDP tunel. Only one transport label is needed.

7. Router 15-P is PHP router and will pop the LDP label:

root@MX:15-P> show route label 299872 

mpls.0: 8 destinations, 8 routes (8 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

299872             *[LDP/9] 01:56:46, metric 1
                    > to 10.10.1.7 via ge-0/0/0.1516, Pop      
299872(S=0)        *[LDP/9] 01:56:46, metric 1
                    > to 10.10.1.7 via ge-0/0/0.1516, Pop      

8. Router 16-PE will receive the traffic with only L3VPN label. It will pop the label and forward the traffic to de destination:

root@MX:16-PE> show route 192.168.0.2 

VPN.inet.0: 5 destinations, 6 routes (5 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

192.168.0.2/32     *[BGP/170] 02:45:28, localpref 100
                      AS path: 1 I, validation-state: unverified
                    > to 192.168.1.2 via ge-0/0/1.216

Documentation