Interprovider L3VPN Option C on a vMX

Implementation Description

In this article, I will build an Interprovider L3VPN Option C setup on a single vMX. Each individual router will be configured as a logical router. The vMX router has a back-to-back connection between ge-0/0/0 and ge-0/0/1 ports. For each different link, a different VLAN number will be configured to separate the traffic generated based on router ID number.

Note: This option is the most scalable solution comparing with option A and option B.

Network Diagram

IP Allocation

  • AS 20: 192.168.20.0/16
    • CE 21: 192.168.0.21/32
    • CE 22: 192.168.0.22/32
    • Link 1121: 192.168.20.0/31
    • Link 1622: 192.168.20.2/31
  • AS 30: 192.168.30.0/16
    • CE 31: 192.168.0.31/32
    • CE 32: 192.168.0.32/32
    • Link 1131: 192.168.30.0/31
    • Link 1632: 192.168.30.2/31
  • AS 100: 10.100.0.0/16
    • PE 11: 10.100.0.11/32
    • P 12: 10.100.0.12/32
    • ASBR 13: 10.100.0.13/32
    • Internal Links: 10.100.1.0/16
  • AS 200: 10.200.0.0/16
    • PE 16: 10.200.0.16/32
    • P 15: 10.200.0.15/32
    • ASBR 14: 10.200.0.14/32
    • Internal Links: 10.200.1.0/16
  • Inter-ASBR link:
    • Link 1314: 172.167.12.0/31

Full Configuration

show | no-more 
## Last changed: 2018-03-07 08:58:37 UTC
version 15.1F4.15;
groups {
    isis {
        logical-systems {
            <*> {
                protocols {
                    isis {
                        level 1 disable;
                        interface <*> {
                            point-to-point;
                        }
                    }
                }
            }
        }
    }
}
apply-groups isis;
system {
    host-name MX;
    root-authentication {
        encrypted-password "$5$L3F31155$kVyagZl2v/WM9s32/hi7VCXxM5o0vupYD.LO3uvCif4"; ## SECRET-DATA
    }
    services {
        ssh;
        netconf {
            ssh;
        }
    }
    syslog {
        user * {
            any emergency;
        }
        file messages {
            any notice;
            authorization info;
        }
        file interactive-commands {
            interactive-commands any;
        }
    }
}
logical-systems {
    11-PE {
        interfaces {
            ge-0/0/0 {
                unit 1112 {
                    vlan-id 1112;
                    family inet {
                        address 10.100.1.0/31;
                    }
                    family iso;
                    family mpls;
                }
                unit 1121 {
                    vlan-id 1121;
                    family inet {
                        address 192.168.20.0/31;
                    }
                }
                unit 1131 {
                    vlan-id 1131;
                    family inet {
                        address 192.168.30.0/31;
                    }
                }
            }
            lo0 {
                unit 11 {
                    family inet {
                        address 10.100.0.11/32;
                    }
                    family iso {
                        address 49.0100.0101.0000.0011.00;
                    }
                }
            }
        }
        protocols {
            mpls {
                interface all;
            }
            bgp {
                group to-AS200 {
                    type external;
                    multihop {
                        ttl 10;
                    }
                    local-address 10.100.0.11;
                    family inet-vpn {
                        unicast;
                    }
                    peer-as 200;
                    neighbor 10.200.0.16;
                }
                group internal {
                    type internal;
                    local-address 10.100.0.11;
                    family inet {
                        labeled-unicast {
                            resolve-vpn;
                        }
                    }
                    neighbor 10.100.0.13;
                }
            }
            isis {
                interface ge-0/0/0.1112;
                interface lo0.11;
            }
            ldp {
                interface all;
            }
        }
        routing-instances {
            Cust-20 {
                instance-type vrf;
                interface ge-0/0/0.1121;
                route-distinguisher 10.100.0.11:20;
                vrf-target target:0.0.100.200:20;
                protocols {
                    bgp {
                        group to-Cust {
                            type external;
                            peer-as 20;
                            as-override;
                            neighbor 192.168.20.1;
                        }
                    }
                }
            }
            Cust-30 {
                instance-type vrf;
                interface ge-0/0/0.1131;
                route-distinguisher 10.100.0.11:30;
                vrf-target target:0.0.100.200:30;
                protocols {
                    bgp {
                        group to-Cust {
                            type external;
                            peer-as 30;
                            as-override;
                            neighbor 192.168.30.1;
                        }
                    }
                }
            }
        }
        routing-options {
            autonomous-system 100;
        }
    }
    12-P {
        interfaces {
            ge-0/0/0 {
                unit 1213 {
                    vlan-id 1213;
                    family inet {
                        address 10.100.1.2/31;
                    }
                    family iso;
                    family mpls;
                }
            }
            ge-0/0/1 {
                unit 1112 {
                    vlan-id 1112;
                    family inet {
                        address 10.100.1.1/31;
                    }
                    family iso;
                    family mpls;
                }
            }
            lo0 {
                unit 12 {
                    family inet {
                        address 10.100.0.12/32;
                    }
                    family iso {
                        address 49.0100.0101.0000.0012.00;
                    }
                }
            }
        }
        protocols {
            mpls {
                traffic-engineering {
                    bgp-igp;
                }
                interface all;
            }
            isis {
                interface all;
            }
            ldp {
                interface all;
            }
        }
        routing-options {
            autonomous-system 100;
        }
    }
    13-ASBR {
        interfaces {
            ge-0/0/0 {
                unit 1314 {
                    vlan-id 1314;
                    family inet {
                        address 172.167.12.0/31;
                    }
                    family mpls;
                }
            }
            ge-0/0/1 {
                unit 1213 {
                    vlan-id 1213;
                    family inet {
                        address 10.100.1.3/31;
                    }
                    family iso;
                    family mpls;
                }
            }
            lo0 {
                unit 13 {
                    family inet {
                        address 10.100.0.13/32;
                    }
                    family iso {
                        address 49.0100.0101.0000.0013.00;
                    }
                }
            }
        }
        protocols {
            mpls {
                traffic-engineering {
                    bgp-igp;
                }
                interface all;
            }
            bgp {
                group to-AS200 {
                    type external;
                    family inet {
                        labeled-unicast;
                    }
                    export to-AS200;
                    peer-as 200;
                    neighbor 172.167.12.1;
                }
                group internal {
                    type internal;
                    local-address 10.100.0.13;
                    family inet {
                        labeled-unicast;
                    }
                    neighbor 10.100.0.11;
                }
            }
            isis {
                interface all;
            }
            ldp {
                interface ge-0/0/1.1213;
                interface lo0.13;
            }
        }
        policy-options {
            policy-statement to-AS200 {
                term PE11-lo0 {
                    from {
                        route-filter 10.100.0.11/32 exact;
                    }
                    then accept;
                }
                term reject {
                    then reject;
                }
            }
        }
        routing-options {
            autonomous-system 100;
        }
    }
    14-ASBR {
        interfaces {
            ge-0/0/0 {
                unit 1415 {
                    vlan-id 1415;
                    family inet {
                        address 10.200.1.4/31;
                    }
                    family iso;
                    family mpls;
                }
            }
            ge-0/0/1 {
                unit 1314 {
                    vlan-id 1314;
                    family inet {
                        address 172.167.12.1/31;
                    }
                    family mpls;
                }
            }
            lo0 {
                unit 14 {
                    family inet {
                        address 10.200.0.14/32;
                    }
                    family iso {
                        address 49.0200.0102.0000.0014.00;
                    }
                }
            }
        }
        protocols {
            mpls {
                traffic-engineering {
                    bgp-igp;
                }
                interface all;
            }
            bgp {
                group to-AS100 {
                    type external;
                    family inet {
                        labeled-unicast;
                    }
                    export to-AS100;
                    peer-as 100;
                    neighbor 172.167.12.0;
                }
                group internal {
                    type internal;
                    local-address 10.200.0.14;
                    family inet {
                        labeled-unicast;
                    }
                    neighbor 10.200.0.16;
                }
            }
            isis {
                interface all;
            }
            ldp {
                interface ge-0/0/0.1415;
                interface lo0.14;
            }
        }
        policy-options {
            policy-statement to-AS100 {
                term PE16-lo0 {
                    from {
                        route-filter 10.200.0.16/32 exact;
                    }
                    then accept;
                }
                term reject {
                    then reject;
                }
            }
        }
        routing-options {
            autonomous-system 200;
        }
    }
    15-P {
        interfaces {
            ge-0/0/0 {
                unit 1516 {
                    vlan-id 1516;
                    family inet {
                        address 10.200.1.6/31;
                    }
                    family iso;
                    family mpls;
                }
            }
            ge-0/0/1 {
                unit 1415 {
                    vlan-id 1415;
                    family inet {
                        address 10.200.1.5/31;
                    }
                    family iso;
                    family mpls;
                }
            }
            lo0 {
                unit 15 {
                    family inet {
                        address 10.200.0.15/32;
                    }
                    family iso {
                        address 49.0200.0102.0000.0015.00;
                    }
                }
            }
        }
        protocols {
            mpls {
                traffic-engineering {
                    bgp-igp;
                }
                interface all;
            }
            isis {
                interface all;
            }
            ldp {
                interface all;
            }
        }
    }
    16-PE {
        interfaces {
            ge-0/0/0 {
                unit 1622 {
                    vlan-id 1622;
                    family inet {
                        address 192.168.20.2/31;
                    }
                }
                unit 1632 {
                    vlan-id 1632;
                    family inet {
                        address 192.168.30.2/31;
                    }
                }
            }
            ge-0/0/1 {
                unit 1516 {
                    vlan-id 1516;
                    family inet {
                        address 10.200.1.7/31;
                    }
                    family iso;
                    family mpls;
                }
            }
            lo0 {
                unit 16 {
                    family inet {
                        address 10.200.0.16/32;
                    }
                    family iso {
                        address 49.0200.0102.0000.0016.00;
                    }
                }
            }
        }
        protocols {
            mpls {
                interface all;
            }
            bgp {
                group internal {
                    type internal;
                    local-address 10.200.0.16;
                    family inet {
                        labeled-unicast {
                            resolve-vpn;
                        }
                    }
                    neighbor 10.200.0.14;
                }
                group to-AS100 {
                    type external;
                    multihop {
                        ttl 10;
                    }
                    local-address 10.200.0.16;
                    family inet-vpn {
                        unicast;
                    }
                    peer-as 100;
                    neighbor 10.100.0.11;
                }
            }
            isis {
                interface all;
            }
            ldp {
                interface all;
            }
        }
        routing-instances {
            Cust-20 {
                instance-type vrf;
                interface ge-0/0/0.1622;
                route-distinguisher 10.100.0.16:20;
                vrf-target target:0.0.100.200:20;
                protocols {
                    bgp {
                        group to-Cust {
                            type external;
                            peer-as 20;
                            as-override;
                            neighbor 192.168.20.3;
                        }
                    }
                }
            }
            Cust-30 {
                instance-type vrf;
                interface ge-0/0/0.1632;
                route-distinguisher 10.100.0.16:30;
                vrf-target target:0.0.100.200:30;
                protocols {
                    bgp {
                        group to-Cust {
                            type external;
                            peer-as 30;
                            as-override;
                            neighbor 192.168.30.3;
                        }
                    }
                }
            }
        }
        routing-options {
            autonomous-system 200;
        }
    }
    21-CE {
        interfaces {
            ge-0/0/1 {
                unit 1121 {
                    vlan-id 1121;
                    family inet {
                        address 192.168.20.1/31;
                    }
                }
            }
            lo0 {
                unit 21 {
                    family inet {
                        address 192.168.0.21/32;
                    }
                }
            }
        }
        protocols {
            bgp {
                group AS100 {
                    type external;
                    export to-bgp;
                    peer-as 100;
                    neighbor 192.168.20.0;
                }
            }
        }
        policy-options {
            policy-statement to-bgp {
                from protocol direct;
                then accept;
            }
        }
        routing-options {
            autonomous-system 20;
        }
    }
    22-CE {
        interfaces {
            ge-0/0/1 {
                unit 1622 {
                    vlan-id 1622;
                    family inet {
                        address 192.168.20.3/31;
                    }
                }
            }
            lo0 {
                unit 22 {
                    family inet {
                        address 192.168.0.22/32;
                    }
                }
            }
        }
        protocols {
            bgp {
                group AS200 {
                    type external;
                    export to-bgp;
                    peer-as 200;
                    neighbor 192.168.20.2;
                }
            }
        }
        policy-options {
            policy-statement to-bgp {
                from protocol direct;
                then accept;
            }
        }
        routing-options {
            autonomous-system 20;
        }
    }
    31-CE {
        interfaces {
            ge-0/0/1 {
                unit 1131 {
                    vlan-id 1131;
                    family inet {
                        address 192.168.30.1/31;
                    }
                }
            }
            lo0 {
                unit 31 {
                    family inet {
                        address 192.168.0.31/32;
                    }
                }
            }
        }
        protocols {
            bgp {
                group AS100 {
                    type external;
                    export to-bgp;
                    peer-as 100;
                    neighbor 192.168.30.0;
                }
            }
        }
        policy-options {
            policy-statement to-bgp {
                from protocol direct;
                then accept;
            }
        }
        routing-options {
            autonomous-system 30;
        }
    }
    32-CE {
        interfaces {
            ge-0/0/1 {
                unit 1632 {
                    vlan-id 1632;
                    family inet {
                        address 192.168.30.3/31;
                    }
                }
            }
            lo0 {
                unit 32 {
                    family inet {
                        address 192.168.0.32/32;
                    }
                }
            }
        }
        protocols {
            bgp {
                group AS200 {
                    type external;
                    export to-bgp;
                    peer-as 200;
                    neighbor 192.168.30.2;
                }
            }
        }
        policy-options {
            policy-statement to-bgp {
                from protocol direct;
                then accept;
            }
        }
        routing-options {
            autonomous-system 30;
        }
    }
}
interfaces {
    ge-0/0/0 {
        description "to ge-0/0/1";
        vlan-tagging;
    }
    ge-0/0/1 {
        description "to ge-0/0/0";
        vlan-tagging;
    }
    ge-0/0/2 {
        description "to ge-0/0/3";
        vlan-tagging;
    }
    ge-0/0/3 {
        description "to ge-0/0/2";
        vlan-tagging;
    }
    ge-0/0/4 {
        description "to ge-0/0/5";
        vlan-tagging;
    }
    ge-0/0/5 {
        description "to ge-0/0/4";
        vlan-tagging;
    }
    ge-0/0/6 {
        description "to ge-0/0/7";
        vlan-tagging;
    }
    ge-0/0/7 {
        description "to ge-0/0/6";
        vlan-tagging;
    }
    ge-0/0/8 {
        description "to eth1";
    }
    ge-0/0/9 {
        description "to eth2";
    }
    fxp0 {
        unit 0 {
            family inet {
                address 192.168.83.11/24;
            }
        }
    }
}

Verification

root@MX:21-CE> show route 

inet.0: 5 destinations, 5 routes (5 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

192.168.0.21/32    *[Direct/0] 02:05:23
                    > via lo0.21
192.168.0.22/32    *[BGP/170] 00:00:33, localpref 100
                      AS path: 100 200 100 I, validation-state: unverified
                    > to 192.168.20.0 via ge-0/0/1.1121
192.168.20.0/31    *[Direct/0] 02:04:45
                    > via ge-0/0/1.1121
192.168.20.1/32    *[Local/0] 02:04:45
                      Local via ge-0/0/1.1121
192.168.20.2/31    *[BGP/170] 00:00:33, localpref 100
                      AS path: 100 200 I, validation-state: unverified
                    > to 192.168.20.0 via ge-0/0/1.1121

root@MX:21-CE> ping 192.168.0.22 count 3 source 192.168.0.21    
PING 192.168.0.22 (192.168.0.22): 56 data bytes
64 bytes from 192.168.0.22: icmp_seq=0 ttl=52 time=11.834 ms
64 bytes from 192.168.0.22: icmp_seq=1 ttl=52 time=11.269 ms
64 bytes from 192.168.0.22: icmp_seq=2 ttl=52 time=8.738 ms

--- 192.168.0.22 ping statistics ---
3 packets transmitted, 3 packets received, 0% packet loss
round-trip min/avg/max/stddev = 8.738/10.614/11.834/1.346 ms

Label Operation

1. Customer router will send simple traffic.

root@MX:21-CE> show route 192.168.0.22 

inet.0: 5 destinations, 5 routes (5 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

192.168.0.22/32    *[BGP/170] 00:05:33, localpref 100
                      AS path: 100 200 100 I, validation-state: unverified
                    > to 192.168.20.0 via ge-0/0/1.1121

2. Router 11-PE will add three labels to the traffic, L3VPN label, BGP-LU label and LDP transport label.

root@MX:11-PE> show route 192.168.0.22    

Cust-20.inet.0: 5 destinations, 6 routes (5 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

192.168.0.22/32    *[BGP/170] 00:10:04, localpref 100, from 10.200.0.16
                      AS path: 200 20 I, validation-state: unverified
                    > to 10.100.1.1 via ge-0/0/0.1112, Push 299856, Push 299952, Push 299824(top)

root@MX:11-PE> show route table mpls      

mpls.0: 9 destinations, 9 routes (9 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

0                  *[MPLS/0] 02:15:03, metric 1
                      Receive
1                  *[MPLS/0] 02:15:03, metric 1
                      Receive
2                  *[MPLS/0] 02:15:03, metric 1
                      Receive
13                 *[MPLS/0] 02:15:03, metric 1
                      Receive
299776             *[LDP/9] 02:14:03, metric 1
                    > to 10.100.1.1 via ge-0/0/0.1112, Pop      
299776(S=0)        *[LDP/9] 02:14:03, metric 1
                    > to 10.100.1.1 via ge-0/0/0.1112, Pop      
299840             *[LDP/9] 00:30:25, metric 1
                    > to 10.100.1.1 via ge-0/0/0.1112, Swap 299824
299888             *[VPN/170] 00:10:08
                    > to 192.168.20.1 via ge-0/0/0.1121, Pop      
299904             *[VPN/170] 00:10:08
                    > to 192.168.30.1 via ge-0/0/0.1131, Pop           

3. Router 12-P is PHP and will pop the LDP label.

root@MX:12-P> show route 192.168.0.22  

root@MX:12-P> show route label 299824 

mpls.0: 8 destinations, 8 routes (8 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

299824             *[LDP/9] 00:31:02, metric 1
                    > to 10.100.1.3 via ge-0/0/0.1213, Pop      
299824(S=0)        *[LDP/9] 00:31:02, metric 1
                    > to 10.100.1.3 via ge-0/0/0.1213, Pop         

4. Router 13-ASBR will swap the BGP-LU label.

root@MX:13-ASBR> show route label 299952 

mpls.0: 9 destinations, 9 routes (9 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

299952             *[VPN/170] 00:15:19
                    > to 172.167.12.1 via ge-0/0/0.1314, Swap 299920

5. Router 14-ASBR will swap BGP-LU with a new LDP label.

root@MX:14-ASBR> show route 10.200.0.16 

inet.0: 10 destinations, 12 routes (10 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

10.200.0.16/32     *[LDP/9] 00:30:22, metric 1
                    > to 10.200.1.5 via ge-0/0/0.1415, Push 299824
                    [IS-IS/18] 00:31:56, metric 20
                    > to 10.200.1.5 via ge-0/0/0.1415

root@MX:14-ASBR> show route label 299920    

mpls.0: 9 destinations, 9 routes (9 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

299920             *[VPN/170] 00:27:37
                    > to 10.200.1.5 via ge-0/0/0.1415, Swap 299824

6. Router 15-P is PHP and will pop the LDP label.

root@MX:15-P> show route 192.168.0.22 

root@MX:15-P> show route label 299824 

mpls.0: 8 destinations, 8 routes (8 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

299824             *[LDP/9] 00:31:21, metric 1
                    > to 10.200.1.7 via ge-0/0/0.1516, Pop      
299824(S=0)        *[LDP/9] 00:31:21, metric 1
                    > to 10.200.1.7 via ge-0/0/0.1516, Pop

7. Router 16-PE will pop the L3VPN label and will forward simple traffic to the end customer.

root@MX:16-PE> show route label 299856 

mpls.0: 9 destinations, 9 routes (9 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

299856             *[VPN/170] 00:20:01
                    > to 192.168.20.3 via ge-0/0/0.1622, Pop      

root@MX:16-PE> show route 192.168.0.22 

Cust-20.inet.0: 5 destinations, 6 routes (5 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

192.168.0.22/32    *[BGP/170] 02:24:35, localpref 100
                      AS path: 20 I, validation-state: unverified
                    > to 192.168.20.3 via ge-0/0/0.1622

Sources:

Interprovider L3VPN Option B on a vMX

Implementation Description

In this article, I will build an Interprovider L3VPN Option B setup on a single vMX. Each individual router will be configured as a logical router. The vMX router has a back-to-back connection between ge-0/0/0 and ge-0/0/1 ports. For each different link, a different VLAN number will be configured to separate the traffic generated based on router ID number.

Network Diagram

IP Allocation

  • AS 20: 192.168.20.0/16
    • CE 21: 192.168.0.21/32
    • CE 22: 192.168.0.22/32
    • Link 1121: 192.168.20.0/31
    • Link 1622: 192.168.20.2/31
  • AS 30: 192.168.30.0/16
    • CE 31: 192.168.0.31/32
    • CE 32: 192.168.0.32/32
    • Link 1131: 192.168.30.0/31
    • Link 1632: 192.168.30.2/31
  • AS 100: 10.100.0.0/16
    • PE 11: 10.100.0.11/32
    • P 12: 10.100.0.12/32
    • ASBR 13: 10.100.0.13/32
    • Internal Links: 10.100.1.0/16
  • AS 200: 10.200.0.0/16
    • PE 16: 10.200.0.16/32
    • P 15: 10.200.0.15/32
    • ASBR 14: 10.200.0.14/32
    • Internal Links: 10.200.1.0/16
  • Inter-ASBR link:
    • Link 1314: 172.167.12.0/31

Full Configuration

## Last changed: 2018-03-07 07:03:07 UTC
version 15.1F4.15;
groups {
    isis {
        logical-systems {
            <*> {
                protocols {
                    isis {
                        level 1 disable;
                        interface <*> {
                            point-to-point;
                        }
                    }
                }
            }
        }
    }
}
apply-groups isis;
system {
    host-name MX;
    root-authentication {
        encrypted-password "$5$L3F31155$kVyagZl2v/WM9s32/hi7VCXxM5o0vupYD.LO3uvCif4"; ## SECRET-DATA
    }
    services {
        ssh;
        netconf {
            ssh;
        }
    }
    syslog {
        user * {
            any emergency;
        }
        file messages {
            any notice;
            authorization info;
        }
        file interactive-commands {
            interactive-commands any;
        }
    }
}
logical-systems {
    11-PE {
        interfaces {
            ge-0/0/0 {
                unit 1112 {
                    vlan-id 1112;
                    family inet {
                        address 10.100.1.0/31;
                    }
                    family iso;
                    family mpls;
                }
                unit 1121 {
                    vlan-id 1121;
                    family inet {
                        address 192.168.20.0/31;
                    }
                }
                unit 1131 {
                    vlan-id 1131;
                    family inet {
                        address 192.168.30.0/31;
                    }
                }
            }
            lo0 {
                unit 11 {
                    family inet {
                        address 10.100.0.11/32;
                    }
                    family iso {
                        address 49.0100.0101.0000.0011.00;
                    }
                }
            }
        }
        protocols {
            mpls {
                interface all;
            }
            bgp {
                group internal {
                    type internal;
                    local-address 10.100.0.11;
                    family inet-vpn {
                        any;
                    }
                    neighbor 10.100.0.13;
                }
            }
            isis {
                interface ge-0/0/0.1112;
                interface lo0.11;
            }
            ldp {
                interface all;
            }
        }
        routing-instances {
            Cust-20 {
                instance-type vrf;
                interface ge-0/0/0.1121;
                route-distinguisher 10.100.0.11:20;
                vrf-target target:0.0.100.200:20;
                protocols {
                    bgp {
                        group to-Cust {
                            type external;
                            peer-as 20;
                            as-override;
                            neighbor 192.168.20.1;
                        }
                    }
                }
            }
            Cust-30 {
                instance-type vrf;
                interface ge-0/0/0.1131;
                route-distinguisher 10.100.0.11:30;
                vrf-target target:0.0.100.200:30;
                protocols {
                    bgp {
                        group to-Cust {
                            type external;
                            peer-as 30;
                            as-override;
                            neighbor 192.168.30.1;
                        }
                    }
                }
            }
        }
        routing-options {
            autonomous-system 100;
        }
    }
    12-P {
        interfaces {
            ge-0/0/0 {
                unit 1213 {
                    vlan-id 1213;
                    family inet {
                        address 10.100.1.2/31;
                    }
                    family iso;
                    family mpls;
                }
            }
            ge-0/0/1 {
                unit 1112 {
                    vlan-id 1112;
                    family inet {
                        address 10.100.1.1/31;
                    }
                    family iso;
                    family mpls;
                }
            }
            lo0 {
                unit 12 {
                    family inet {
                        address 10.100.0.12/32;
                    }
                    family iso {
                        address 49.0100.0101.0000.0012.00;
                    }
                }
            }
        }
        protocols {
            mpls {
                interface all;
            }
            isis {
                interface all;
            }
            ldp {
                interface all;
            }
        }
        routing-options {
            autonomous-system 100;
        }
    }
    13-ASBR {
        interfaces {
            ge-0/0/0 {
                unit 1314 {
                    vlan-id 1314;
                    family inet {
                        address 172.167.12.0/31;
                    }
                    family mpls;
                }
            }
            ge-0/0/1 {
                unit 1213 {
                    vlan-id 1213;
                    family inet {
                        address 10.100.1.3/31;
                    }
                    family iso;
                    family mpls;
                }
            }
            lo0 {
                unit 13 {
                    family inet {
                        address 10.100.0.13/32;
                    }
                    family iso {
                        address 49.0100.0101.0000.0013.00;
                    }
                }
            }
        }
        protocols {
            mpls {
                interface all;
            }
            bgp {
                group internal {
                    type internal;
                    local-address 10.100.0.13;
                    family inet-vpn {
                        any;
                    }
                    neighbor 10.100.0.11;
                }
                group to-AS200 {
                    type external;
                    family inet-vpn {
                        any;
                    }
                    peer-as 200;
                    neighbor 172.167.12.1;
                }
            }
            isis {
                interface all;
            }
            ldp {
                interface ge-0/0/1.1213;
                interface lo0.13;
            }
        }
        routing-options {
            autonomous-system 100;
        }
    }
    14-ASBR {
        interfaces {
            ge-0/0/0 {
                unit 1415 {
                    vlan-id 1415;
                    family inet {
                        address 10.200.1.4/31;
                    }
                    family iso;
                    family mpls;
                }
            }
            ge-0/0/1 {
                unit 1314 {
                    vlan-id 1314;
                    family inet {
                        address 172.167.12.1/31;
                    }
                    family mpls;
                }
            }
            lo0 {
                unit 14 {
                    family inet {
                        address 10.200.0.14/32;
                    }
                    family iso {
                        address 49.0200.0102.0000.0014.00;
                    }
                }
            }
        }
        protocols {
            mpls {
                interface all;
            }
            bgp {
                group internal {
                    type internal;
                    local-address 10.200.0.14;
                    family inet-vpn {
                        any;
                    }
                    neighbor 10.200.0.16;
                }
                group to-AS100 {
                    type external;
                    family inet-vpn {
                        any;
                    }
                    peer-as 100;
                    neighbor 172.167.12.0;
                }
            }
            isis {
                interface all;
            }
            ldp {
                interface ge-0/0/0.1415;
                interface lo0.14;
            }
        }
        routing-options {
            autonomous-system 200;
        }
    }
    15-P {
        interfaces {
            ge-0/0/0 {
                unit 1516 {
                    vlan-id 1516;
                    family inet {
                        address 10.200.1.6/31;
                    }
                    family iso;
                    family mpls;
                }
            }
            ge-0/0/1 {
                unit 1415 {
                    vlan-id 1415;
                    family inet {
                        address 10.200.1.5/31;
                    }
                    family iso;
                    family mpls;
                }
            }
            lo0 {
                unit 15 {
                    family inet {
                        address 10.200.0.15/32;
                    }
                    family iso {
                        address 49.0200.0102.0000.0015.00;
                    }
                }
            }
        }
        protocols {
            mpls {
                interface all;
            }
            isis {
                interface all;
            }
            ldp {
                interface all;
            }
        }
    }
    16-PE {
        interfaces {
            ge-0/0/0 {
                unit 1622 {
                    vlan-id 1622;
                    family inet {
                        address 192.168.20.2/31;
                    }
                }
                unit 1632 {
                    vlan-id 1632;
                    family inet {
                        address 192.168.30.2/31;
                    }
                }
            }
            ge-0/0/1 {
                unit 1516 {
                    vlan-id 1516;
                    family inet {
                        address 10.200.1.7/31;
                    }
                    family iso;
                    family mpls;
                }
            }
            lo0 {
                unit 16 {
                    family inet {
                        address 10.200.0.16/32;
                    }
                    family iso {
                        address 49.0200.0102.0000.0016.00;
                    }
                }
            }
        }
        protocols {
            mpls {
                interface all;
            }
            bgp {
                group internal {
                    type internal;
                    local-address 10.200.0.16;
                    family inet-vpn {
                        any;
                    }
                    neighbor 10.200.0.14;
                }
            }
            isis {
                interface all;
            }
            ldp {
                interface all;
            }
        }
        routing-instances {
            Cust-20 {
                instance-type vrf;
                interface ge-0/0/0.1622;
                route-distinguisher 10.100.0.16:20;
                vrf-target target:0.0.100.200:20;
                protocols {
                    bgp {
                        group to-Cust {
                            type external;
                            peer-as 20;
                            as-override;
                            neighbor 192.168.20.3;
                        }
                    }
                }
            }
            Cust-30 {
                instance-type vrf;
                interface ge-0/0/0.1632;
                route-distinguisher 10.100.0.16:30;
                vrf-target target:0.0.100.200:30;
                protocols {
                    bgp {
                        group to-Cust {
                            type external;
                            peer-as 30;
                            as-override;
                            neighbor 192.168.30.3;
                        }
                    }
                }
            }
        }
        routing-options {
            autonomous-system 200;
        }
    }
    21-CE {
        interfaces {
            ge-0/0/1 {
                unit 1121 {
                    vlan-id 1121;
                    family inet {
                        address 192.168.20.1/31;
                    }
                }
            }
            lo0 {
                unit 21 {
                    family inet {
                        address 192.168.0.21/32;
                    }
                }
            }
        }
        protocols {
            bgp {
                group AS100 {
                    type external;
                    export to-bgp;
                    peer-as 100;
                    neighbor 192.168.20.0;
                }
            }
        }
        policy-options {
            policy-statement to-bgp {
                from protocol direct;
                then accept;
            }
        }
        routing-options {
            autonomous-system 20;
        }
    }
    22-CE {
        interfaces {
            ge-0/0/1 {
                unit 1622 {
                    vlan-id 1622;
                    family inet {
                        address 192.168.20.3/31;
                    }
                }
            }
            lo0 {
                unit 22 {
                    family inet {
                        address 192.168.0.22/32;
                    }
                }
            }
        }
        protocols {
            bgp {
                group AS200 {
                    type external;
                    export to-bgp;
                    peer-as 200;
                    neighbor 192.168.20.2;
                }
            }
        }
        policy-options {
            policy-statement to-bgp {
                from protocol direct;
                then accept;
            }
        }
        routing-options {
            autonomous-system 20;
        }
    }
    31-CE {
        interfaces {
            ge-0/0/1 {
                unit 1131 {
                    vlan-id 1131;
                    family inet {
                        address 192.168.30.1/31;
                    }
                }
            }
            lo0 {
                unit 31 {
                    family inet {
                        address 192.168.0.31/32;
                    }
                }
            }
        }
        protocols {
            bgp {
                group AS100 {
                    type external;
                    export to-bgp;
                    peer-as 100;
                    neighbor 192.168.30.0;
                }
            }
        }
        policy-options {
            policy-statement to-bgp {
                from protocol direct;
                then accept;
            }
        }
        routing-options {
            autonomous-system 30;
        }
    }
    32-CE {
        interfaces {
            ge-0/0/1 {
                unit 1632 {
                    vlan-id 1632;
                    family inet {
                        address 192.168.30.3/31;
                    }
                }
            }
            lo0 {
                unit 32 {
                    family inet {
                        address 192.168.0.32/32;
                    }
                }
            }
        }
        protocols {
            bgp {
                group AS200 {
                    type external;
                    export to-bgp;
                    peer-as 200;
                    neighbor 192.168.30.2;
                }
            }
        }
        policy-options {
            policy-statement to-bgp {
                from protocol direct;
                then accept;
            }
        }
        routing-options {
            autonomous-system 30;
        }
    }
}
interfaces {
    ge-0/0/0 {
        description "to ge-0/0/1";
        vlan-tagging;
    }
    ge-0/0/1 {
        description "to ge-0/0/0";
        vlan-tagging;
    }
    ge-0/0/2 {
        description "to ge-0/0/3";
        vlan-tagging;
    }
    ge-0/0/3 {
        description "to ge-0/0/2";
        vlan-tagging;
    }
    ge-0/0/4 {
        description "to ge-0/0/5";
        vlan-tagging;
    }
    ge-0/0/5 {
        description "to ge-0/0/4";
        vlan-tagging;
    }
    ge-0/0/6 {
        description "to ge-0/0/7";
        vlan-tagging;
    }
    ge-0/0/7 {
        description "to ge-0/0/6";
        vlan-tagging;
    }
    ge-0/0/8 {
        description "to eth1";
    }
    ge-0/0/9 {
        description "to eth2";
    }
    fxp0 {
        unit 0 {
            family inet {
                address 192.168.83.11/24;
            }
        }
    }
}

Verification

root@MX:21-CE> show route    

inet.0: 5 destinations, 5 routes (5 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

192.168.0.21/32    *[Direct/0] 00:09:22
                    > via lo0.21
192.168.0.22/32    *[BGP/170] 00:00:14, localpref 100
                      AS path: 100 200 100 I, validation-state: unverified
                    > to 192.168.20.0 via ge-0/0/1.1121
192.168.20.0/31    *[Direct/0] 00:08:44
                    > via ge-0/0/1.1121
192.168.20.1/32    *[Local/0] 00:08:44
                      Local via ge-0/0/1.1121
192.168.20.2/31    *[BGP/170] 00:00:13, localpref 100
                      AS path: 100 200 I, validation-state: unverified
                    > to 192.168.20.0 via ge-0/0/1.1121

root@MX:21-CE> ping 192.168.0.22 count 3 source 192.168.0.21  
PING 192.168.0.22 (192.168.0.22): 56 data bytes
64 bytes from 192.168.0.22: icmp_seq=0 ttl=52 time=11.001 ms
64 bytes from 192.168.0.22: icmp_seq=1 ttl=52 time=11.744 ms
64 bytes from 192.168.0.22: icmp_seq=2 ttl=52 time=12.737 ms

--- 192.168.0.22 ping statistics ---
3 packets transmitted, 3 packets received, 0% packet loss
round-trip min/avg/max/stddev = 11.001/11.827/12.737/0.711 ms

Label Operation

1. Customer router will send simple traffic.

root@MX:21-CE> show route 192.168.0.22 

inet.0: 5 destinations, 5 routes (5 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

192.168.0.22/32    *[BGP/170] 00:05:25, localpref 100
                      AS path: 100 200 100 I, validation-state: unverified
                    > to 192.168.20.0 via ge-0/0/1.1121

2. Router 11-PE will add two labels to the traffic, L3VPN label and LDP transport label.

root@MX:11-PE> show route 192.168.0.22 

Cust-20.inet.0: 5 destinations, 6 routes (5 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

192.168.0.22/32    *[BGP/170] 00:06:06, localpref 100, from 10.100.0.13
                      AS path: 200 20 I, validation-state: unverified
                    > to 10.100.1.1 via ge-0/0/0.1112, Push 299840, Push 299792(top)

root@MX:11-PE> show route table mpls 

mpls.0: 9 destinations, 9 routes (9 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

0                  *[MPLS/0] 00:16:38, metric 1
                      Receive
1                  *[MPLS/0] 00:16:38, metric 1
                      Receive
2                  *[MPLS/0] 00:16:38, metric 1
                      Receive
13                 *[MPLS/0] 00:16:38, metric 1
                      Receive
299776             *[LDP/9] 00:15:38, metric 1
                    > to 10.100.1.1 via ge-0/0/0.1112, Pop      
299776(S=0)        *[LDP/9] 00:15:38, metric 1
                    > to 10.100.1.1 via ge-0/0/0.1112, Pop      
299792             *[LDP/9] 00:15:13, metric 1
                    > to 10.100.1.1 via ge-0/0/0.1112, Swap 299792
299808             *[VPN/170] 00:14:21
                    > to 192.168.20.1 via ge-0/0/0.1121, Pop      
299824             *[VPN/170] 00:14:21
                    > to 192.168.30.1 via ge-0/0/0.1131, Pop      

3. Router 12-P is PHP and will pop the LDP label.

root@MX:12-P> show route 192.168.0.22  

root@MX:12-P> show route label 299792 

root@MX:12-P> show route label 299792 

mpls.0: 8 destinations, 8 routes (8 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

299792             *[LDP/9] 00:16:25, metric 1
                    > to 10.100.1.3 via ge-0/0/0.1213, Pop      
299792(S=0)        *[LDP/9] 00:16:25, metric 1
                    > to 10.100.1.3 via ge-0/0/0.1213, Pop      

4. Router 13-ASBR will swap the L3VPN label.

root@MX:13-ASBR> show route table bgp.l3vpn.0                                  

bgp.l3vpn.0: 8 destinations, 8 routes (8 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

[...]
10.100.0.16:20:192.168.0.22/32                
                   *[BGP/170] 00:11:44, localpref 100
                      AS path: 200 20 I, validation-state: unverified
                    > to 172.167.12.1 via ge-0/0/0.1314, Push 299808
[...]


root@MX:13-ASBR> show route label 299840 

mpls.0: 11 destinations, 11 routes (11 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

299840             *[VPN/170] 00:21:11
                    > to 172.167.12.1 via ge-0/0/0.1314, Swap 299808

5. Router 14-ASBR will swap L3VPN label and will push LDP label.

root@MX:14-ASBR> show route table bgp.l3vpn.0 

bgp.l3vpn.0: 8 destinations, 8 routes (8 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

[...]
10.100.0.16:20:192.168.0.22/32                
                   *[BGP/170] 00:20:30, localpref 100, from 10.200.0.16
                      AS path: 20 I, validation-state: unverified
                    > to 10.200.1.5 via ge-0/0/0.1415, Push 299808, Push 299776(top)
[...]

root@MX:14-ASBR> show route label 299808                     

mpls.0: 11 destinations, 11 routes (11 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

299808             *[VPN/170] 00:27:53, metric2 1, from 10.200.0.16
                    > to 10.200.1.5 via ge-0/0/0.1415, Swap 299808, Push 299776(top)

6. Router 15-P is PHP and will pop the LDP label.

root@MX:15-P> show route 192.168.0.22 

root@MX:15-P> show route label 299776    

mpls.0: 8 destinations, 8 routes (8 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

299776             *[LDP/9] 00:33:15, metric 1
                    > to 10.200.1.7 via ge-0/0/0.1516, Pop      
299776(S=0)        *[LDP/9] 00:33:15, metric 1
                    > to 10.200.1.7 via ge-0/0/0.1516, Pop 

7. Router 16-PE will pop L3VPN label and will forward simple traffic to the end customer.

root@MX:16-PE> show route label 299808 

mpls.0: 9 destinations, 9 routes (9 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

299808             *[VPN/170] 00:33:13
                    > to 192.168.20.3 via ge-0/0/0.1622, Pop      

root@MX:16-PE> show route 192.168.0.22    

Cust-20.inet.0: 5 destinations, 6 routes (5 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

192.168.0.22/32    *[BGP/170] 00:34:52, localpref 100
                      AS path: 20 I, validation-state: unverified
                    > to 192.168.20.3 via ge-0/0/0.1622

The Factors that Limit the Scalability of this Method

  • In this solution, ASBR routers keep all VPN-IPv4 routes in the routing information base (RIB), and the labels associated with the prefixes are kept in the forwarding information base (FIB). Because the RIB and FIB tables can take too much of the respective allocated memory, this solution is not very scalable for an interprovider VPN.
  • If a transit SP is used between SP1 and SP2, the transit SP also has to keep all VPN-IPv4 routes in the RIB and the corresponding labels in the FIB. The ASBRs at the transit SP have the same functionality as ASBRs at SP1 or SP2 in this solution.

Sources:

 

Interprovider L3VPN Option A on a vMX

Implementation Description

In this article, I will build an Interprovider L3VPN Option A setup on a single vMX. Each individual router will be configured as a logical router. The vMX router has a back-to-back connection between ge-0/0/0 and ge-0/0/1 ports. For each different link, a different VLAN number will be configured to separate the traffic generated based on router ID number.

Network Diagram

IP Allocation

  • AS 20: 192.168.20.0/16
    • CE 21: 192.168.0.21/32
    • CE 22: 192.168.0.22/32
    • Link 1121: 192.168.20.0/31
    • Link 1622: 192.168.20.2/31
  • AS 30: 192.168.30.0/16
    • CE 31: 192.168.0.31/32
    • CE 32: 192.168.0.32/32
    • Link 1131: 192.168.30.0/31
    • Link 1632: 192.168.30.2/31
  • AS 100: 10.100.0.0/16
    • PE 11: 10.100.0.11/32
    • P 12: 10.100.0.12/32
    • ASBR 13: 10.100.0.13/32
    • Internal Links: 10.100.1.0/16
  • AS 200: 10.200.0.0/16
    • PE 16: 10.200.0.16/32
    • P 15: 10.200.0.15/32
    • ASBR 14: 10.200.0.14/32
    • Internal Links: 10.200.1.0/16
  • Inter-ASBR links:
    • Link 20: 172.167.20.0/31
    • LInk 30: 172.167.30.0/31

Full Configuration

## Last changed: 2018-03-06 12:52:17 UTC
version 15.1F4.15;
groups {
    isis {
        logical-systems {
            <*> {
                protocols {
                    isis {
                        level 1 disable;
                        interface <*> {
                            point-to-point;
                        }
                    }
                }
            }
        }
    }
}
apply-groups isis;
system {
    host-name MX;
    root-authentication {
        encrypted-password "$5$L3F31155$kVyagZl2v/WM9s32/hi7VCXxM5o0vupYD.LO3uvCif4"; ## SECRET-DATA
    }
    services {
        ssh;
        netconf {
            ssh;
        }
    }
    syslog {
        user * {
            any emergency;
        }
        file messages {
            any notice;
            authorization info;
        }
        file interactive-commands {
            interactive-commands any;
        }
    }
}
logical-systems {
    11-PE {
        interfaces {
            ge-0/0/0 {
                unit 1112 {
                    vlan-id 1112;
                    family inet {
                        address 10.100.1.0/31;
                    }
                    family iso;
                    family mpls;
                }
                unit 1121 {
                    vlan-id 1121;
                    family inet {
                        address 192.168.20.0/31;
                    }
                }
                unit 1131 {
                    vlan-id 1131;
                    family inet {
                        address 192.168.30.0/31;
                    }
                }
            }
            lo0 {
                unit 11 {
                    family inet {
                        address 10.100.0.11/32;
                    }
                    family iso {
                        address 49.0100.0101.0000.0011.00;
                    }
                }
            }
        }
        protocols {
            mpls {
                interface all;
            }
            bgp {
                group internal {
                    type internal;
                    local-address 10.100.0.11;
                    family inet-vpn {
                        any;
                    }
                    neighbor 10.100.0.13;
                }
            }
            isis {
                interface ge-0/0/0.1112;
                interface lo0.11;
            }
            ldp {
                interface all;
            }
        }
        routing-instances {
            Cust-20 {
                instance-type vrf;
                interface ge-0/0/0.1121;
                route-distinguisher 10.100.0.11:20;
                vrf-target target:100:20;
                protocols {
                    bgp {
                        group to-Cust {
                            type external;
                            peer-as 20;
                            as-override;
                            neighbor 192.168.20.1;
                        }
                    }
                }
            }
            Cust-30 {
                instance-type vrf;
                interface ge-0/0/0.1131;
                route-distinguisher 10.100.0.11:30;
                vrf-target target:100:30;
                protocols {
                    bgp {
                        group to-Cust {
                            type external;
                            peer-as 30;
                            as-override;
                            neighbor 192.168.30.1;
                        }
                    }
                }
            }
        }
        routing-options {
            autonomous-system 100;
        }
    }
    12-P {
        interfaces {
            ge-0/0/0 {
                unit 1213 {
                    vlan-id 1213;
                    family inet {
                        address 10.100.1.2/31;
                    }
                    family iso;
                    family mpls;
                }
            }
            ge-0/0/1 {
                unit 1112 {
                    vlan-id 1112;
                    family inet {
                        address 10.100.1.1/31;
                    }
                    family iso;
                    family mpls;
                }
            }
            lo0 {
                unit 12 {
                    family inet {
                        address 10.100.0.12/32;
                    }
                    family iso {
                        address 49.0100.0101.0000.0012.00;
                    }
                }
            }
        }
        protocols {
            mpls {
                interface all;
            }
            isis {
                interface all;
            }
            ldp {
                interface all;
            }
        }
        routing-options {
            autonomous-system 100;
        }
    }
    13-ASBR {
        interfaces {
            ge-0/0/0 {
                unit 20 {
                    vlan-id 20;
                    family inet {
                        address 172.167.20.0/31;
                    }
                }
                unit 30 {
                    vlan-id 30;
                    family inet {
                        address 172.167.30.0/31;
                    }
                }
            }
            ge-0/0/1 {
                unit 1213 {
                    vlan-id 1213;
                    family inet {
                        address 10.100.1.3/31;
                    }
                    family iso;
                    family mpls;
                }
            }
            lo0 {
                unit 13 {
                    family inet {
                        address 10.100.0.13/32;
                    }
                    family iso {
                        address 49.0100.0101.0000.0013.00;
                    }
                }
            }
        }
        protocols {
            mpls {
                interface all;
            }
            bgp {
                group internal {
                    type internal;
                    local-address 10.100.0.13;
                    family inet-vpn {
                        any;
                    }
                    neighbor 10.100.0.11;
                }
            }
            isis {
                interface all;
            }
            ldp {
                interface all;
            }
        }
        routing-instances {
            Cust-20 {
                instance-type vrf;
                interface ge-0/0/0.20;
                route-distinguisher 10.100.0.13:20;
                vrf-target target:100:20;
                protocols {
                    bgp {
                        group to-AS200 {
                            type external;
                            peer-as 200;
                            neighbor 172.167.20.1;
                        }
                    }
                }
            }
            Cust-30 {
                instance-type vrf;
                interface ge-0/0/0.30;
                route-distinguisher 10.100.0.13:30;
                vrf-target target:100:30;
                protocols {
                    bgp {
                        group to-AS200 {
                            type external;
                            peer-as 200;
                            neighbor 172.167.30.1;
                        }
                    }
                }
            }
        }
        routing-options {
            autonomous-system 100;
        }
    }
    14-ASBR {
        interfaces {
            ge-0/0/0 {
                unit 1415 {
                    vlan-id 1415;
                    family inet {
                        address 10.200.1.4/31;
                    }
                    family iso;
                    family mpls;
                }
            }
            ge-0/0/1 {
                unit 20 {
                    vlan-id 20;
                    family inet {
                        address 172.167.20.1/31;
                    }
                }
                unit 30 {
                    vlan-id 30;
                    family inet {
                        address 172.167.30.1/31;
                    }
                }
            }
            lo0 {
                unit 14 {
                    family inet {
                        address 10.200.0.14/32;
                    }
                    family iso {
                        address 49.0200.0102.0000.0014.00;
                    }
                }
            }
        }
        protocols {
            mpls {
                interface all;
            }
            bgp {
                group internal {
                    type internal;
                    local-address 10.200.0.14;
                    family inet-vpn {
                        any;
                    }
                    neighbor 10.200.0.16;
                }
            }
            isis {
                interface all;
            }
            ldp {
                interface all;
            }
        }
        routing-instances {
            Cust-20 {
                instance-type vrf;
                interface ge-0/0/1.20;
                route-distinguisher 10.200.0.13:20;
                vrf-target target:200:20;
                protocols {
                    bgp {
                        group to-AS100 {
                            type external;
                            peer-as 100;
                            neighbor 172.167.20.0;
                        }
                    }
                }
            }
            Cust-30 {
                instance-type vrf;
                interface ge-0/0/1.30;
                route-distinguisher 10.200.0.14:30;
                vrf-target target:200:30;
                protocols {
                    bgp {
                        group to-AS100 {
                            type external;
                            peer-as 100;
                            neighbor 172.167.30.0;
                        }
                    }
                }
            }
        }
        routing-options {
            autonomous-system 200;
        }
    }
    15-P {
        interfaces {
            ge-0/0/0 {
                unit 1516 {
                    vlan-id 1516;
                    family inet {
                        address 10.200.1.6/31;
                    }
                    family iso;
                    family mpls;
                }
            }
            ge-0/0/1 {
                unit 1415 {
                    vlan-id 1415;
                    family inet {
                        address 10.200.1.5/31;
                    }
                    family iso;
                    family mpls;
                }
            }
            lo0 {
                unit 15 {
                    family inet {
                        address 10.200.0.15/32;
                    }
                    family iso {
                        address 49.0100.0102.0000.0015.00;
                    }
                }
            }
        }
        protocols {
            mpls {
                interface all;
            }
            isis {
                interface all;
            }
            ldp {
                interface all;
            }
        }
    }
    16-PE {
        interfaces {
            ge-0/0/0 {
                unit 1622 {
                    vlan-id 1622;
                    family inet {
                        address 192.168.20.2/31;
                    }
                }
                unit 1632 {
                    vlan-id 1632;
                    family inet {
                        address 192.168.30.2/31;
                    }
                }
            }
            ge-0/0/1 {
                unit 1516 {
                    vlan-id 1516;
                    family inet {
                        address 10.200.1.7/31;
                    }
                    family iso;
                    family mpls;
                }
            }
            lo0 {
                unit 16 {
                    family inet {
                        address 10.200.0.16/32;
                    }
                    family iso {
                        address 49.0200.0102.0000.0016.00;
                    }
                }
            }
        }
        protocols {
            mpls {
                interface all;
            }
            bgp {
                group internal {
                    type internal;
                    local-address 10.200.0.16;
                    family inet-vpn {
                        any;
                    }
                    neighbor 10.200.0.14;
                }
            }
            isis {
                interface all;
            }
            ldp {
                interface all;
            }
        }
        routing-instances {
            Cust-20 {
                instance-type vrf;
                interface ge-0/0/0.1622;
                route-distinguisher 10.100.0.16:20;
                vrf-target target:200:20;
                protocols {
                    bgp {
                        group to-Cust {
                            type external;
                            peer-as 20;
                            as-override;
                            neighbor 192.168.20.3;
                        }
                    }
                }
            }
            Cust-30 {
                instance-type vrf;
                interface ge-0/0/0.1632;
                route-distinguisher 10.100.0.16:30;
                vrf-target target:200:30;
                protocols {
                    bgp {
                        group to-Cust {
                            type external;
                            peer-as 30;
                            as-override;
                            neighbor 192.168.30.3;
                        }
                    }
                }
            }
        }
        routing-options {
            autonomous-system 200;
        }
    }
    21-CE {
        interfaces {
            ge-0/0/1 {
                unit 1121 {
                    vlan-id 1121;
                    family inet {
                        address 192.168.20.1/31;
                    }
                }
            }
            lo0 {
                unit 21 {
                    family inet {
                        address 192.168.0.21/32;
                    }
                }
            }
        }
        protocols {
            bgp {
                group AS100 {
                    type external;
                    export to-bgp;
                    peer-as 100;
                    neighbor 192.168.20.0;
                }
            }
        }
        policy-options {
            policy-statement to-bgp {
                from protocol direct;
                then accept;
            }
        }
        routing-options {
            autonomous-system 20;
        }
    }
    22-CE {
        interfaces {
            ge-0/0/1 {
                unit 1622 {
                    vlan-id 1622;
                    family inet {
                        address 192.168.20.3/31;
                    }
                }
            }
            lo0 {
                unit 22 {
                    family inet {
                        address 192.168.0.22/32;
                    }
                }
            }
        }
        protocols {
            bgp {
                group AS200 {
                    type external;
                    export to-bgp;
                    peer-as 200;
                    neighbor 192.168.20.2;
                }
            }
        }
        policy-options {
            policy-statement to-bgp {
                from protocol direct;
                then accept;
            }
        }
        routing-options {
            autonomous-system 20;
        }
    }
    31-CE {
        interfaces {
            ge-0/0/1 {
                unit 1131 {
                    vlan-id 1131;
                    family inet {
                        address 192.168.30.1/31;
                    }
                }
            }
            lo0 {
                unit 31 {
                    family inet {
                        address 192.168.0.31/32;
                    }
                }
            }
        }
        protocols {
            bgp {
                group AS100 {
                    type external;
                    export to-bgp;
                    peer-as 100;
                    neighbor 192.168.30.0;
                }
            }
        }
        policy-options {
            policy-statement to-bgp {
                from protocol direct;
                then accept;
            }
        }
        routing-options {
            autonomous-system 30;
        }
    }
    32-CE {
        interfaces {
            ge-0/0/1 {
                unit 1632 {
                    vlan-id 1632;
                    family inet {
                        address 192.168.30.3/31;
                    }
                }
            }
            lo0 {
                unit 32 {
                    family inet {
                        address 192.168.0.32/32;
                    }
                }
            }
        }
        protocols {
            bgp {
                group AS200 {
                    type external;
                    export to-bgp;
                    peer-as 200;
                    neighbor 192.168.30.2;
                }
            }
        }
        policy-options {
            policy-statement to-bgp {
                from protocol direct;
                then accept;
            }
        }
        routing-options {
            autonomous-system 30;
        }
    }
}
interfaces {
    ge-0/0/0 {
        description "to ge-0/0/1";
        vlan-tagging;
    }
    ge-0/0/1 {
        description "to ge-0/0/0";
        vlan-tagging;
    }
    ge-0/0/2 {
        description "to ge-0/0/3";
        vlan-tagging;
    }
    ge-0/0/3 {
        description "to ge-0/0/2";
        vlan-tagging;
    }
    ge-0/0/4 {
        description "to ge-0/0/5";
        vlan-tagging;
    }
    ge-0/0/5 {
        description "to ge-0/0/4";
        vlan-tagging;
    }
    ge-0/0/6 {
        description "to ge-0/0/7";
        vlan-tagging;
    }
    ge-0/0/7 {
        description "to ge-0/0/6";
        vlan-tagging;
    }
    ge-0/0/8 {
        description "to eth1";
    }
    ge-0/0/9 {
        description "to eth2";
    }
    fxp0 {
        unit 0 {
            family inet {
                address 192.168.83.11/24;
            }
        }
    }
}

Verification

[edit]
root@MX:21-CE# run show route 

inet.0: 6 destinations, 6 routes (6 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

172.167.20.0/31    *[BGP/170] 00:04:04, localpref 100
                      AS path: 100 I, validation-state: unverified
                    > to 192.168.20.0 via ge-0/0/1.1121
192.168.0.21/32    *[Direct/0] 04:10:00
                    > via lo0.21
192.168.0.22/32    *[BGP/170] 00:04:04, localpref 100
                      AS path: 100 200 100 I, validation-state: unverified
                    > to 192.168.20.0 via ge-0/0/1.1121
192.168.20.0/31    *[Direct/0] 04:09:08
                    > via ge-0/0/1.1121
192.168.20.1/32    *[Local/0] 04:09:08
                      Local via ge-0/0/1.1121
192.168.20.2/31    *[BGP/170] 00:04:04, localpref 100
                      AS path: 100 200 I, validation-state: unverified
                    > to 192.168.20.0 via ge-0/0/1.1121

[edit]
root@MX:21-CE# run ping 192.168.0.22 count 3 source 192.168.0.21 
PING 192.168.0.22 (192.168.0.22): 56 data bytes
64 bytes from 192.168.0.22: icmp_seq=0 ttl=52 time=9.114 ms
64 bytes from 192.168.0.22: icmp_seq=1 ttl=52 time=10.155 ms
64 bytes from 192.168.0.22: icmp_seq=2 ttl=52 time=8.088 ms

--- 192.168.0.22 ping statistics ---
3 packets transmitted, 3 packets received, 0% packet loss
round-trip min/avg/max/stddev = 8.088/9.119/10.155/0.844 ms

Label Operation


1. Customer router will send simple traffic.

root@MX:21-CE> show route 192.168.0.22 

inet.0: 6 destinations, 6 routes (6 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

192.168.0.22/32    *[BGP/170] 00:10:08, localpref 100
                      AS path: 100 200 100 I, validation-state: unverified
                    > to 192.168.20.0 via ge-0/0/1.1121

2. Router 11-PE will add two labels to the traffic, L3VPN label and LDP transport label.

root@MX:11-PE> show route 192.168.0.22 

Cust-20.inet.0: 6 destinations, 7 routes (6 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

192.168.0.22/32    *[BGP/170] 00:11:04, localpref 100, from 10.100.0.13
                      AS path: 200 20 I, validation-state: unverified
                    > to 10.100.1.1 via ge-0/0/0.1112, Push 299808, Push 299792(top)

3. Router 12-P is PHP and will pop the LDP label.

root@MX:12-P> show route 192.168.0.22 

root@MX:12-P> show route label 299792 

mpls.0: 8 destinations, 8 routes (8 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

299792             *[LDP/9] 02:34:46, metric 1
                    > to 10.100.1.3 via ge-0/0/0.1213, Pop      
299792(S=0)        *[LDP/9] 02:34:46, metric 1
                    > to 10.100.1.3 via ge-0/0/0.1213, Pop

4. Router 13-ASBR will pop L3VPN label and will forward simple traffic on the dedicated circuit.

root@MX:13-ASBR> show route 192.168.0.22 

Cust-20.inet.0: 6 destinations, 6 routes (6 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

192.168.0.22/32    *[BGP/170] 00:14:39, localpref 100
                      AS path: 200 20 I, validation-state

5. Router 14-ASBR will add two labels to the traffic, L3VPN label and LDP transport label.

root@MX:14-ASBR> show route 192.168.0.22 

Cust-20.inet.0: 6 destinations, 6 routes (6 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

192.168.0.22/32    *[BGP/170] 00:16:04, localpref 100, from 10.200.0.16
                      AS path: 20 I, validation-state: unverified
                    > to 10.200.1.5 via ge-0/0/0.1415, Push 299808, Push 299792(top)

6. Router 15-P is PHP and will pop the LDP label.

root@MX:15-P> show route 192.168.0.22 

root@MX:15-P> show route label 299792 

mpls.0: 8 destinations, 8 routes (8 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

299792             *[LDP/9] 00:23:26, metric 1
                    > to 10.200.1.7 via ge-0/0/0.1516, Pop      
299792(S=0)        *[LDP/9] 00:23:26, metric 1
                    > to 10.200.1.7 via ge-0/0/0.1516, Pop

7. Router 16-PE will pop L3VPN label and will forward simple traffic to the end customer.

root@MX:16-PE> show route 192.168.0.22 

Cust-20.inet.0: 6 destinations, 7 routes (6 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

192.168.0.22/32    *[BGP/170] 00:19:35, localpref 100
                      AS path: 20 I, validation-state: unverified
                    > to 192.168.20.3 via ge-0/0/0.1622

The Factors that Limit the Scalability of this Method

  • All inter-AS VPN routes (potentially a very large number) must be stored in the BGP RIBs and IP routing tables on the AS boundary routers.
  • You must configure VRFs on each AS boundary router.

Sources:

Carrier-of-Carriers with Internet Service Provider as the Customer on a vMX

Implementation description

In this article, I will build a Carrier-of-Carriers with Internet Service Provider as the customer setup on a single vMX. Each individual router will be configured as a logical router. The vMX router has a back-to-back connection between ge-0/0/0 and ge-0/0/1 ports. For each different link, a different VLAN number will be configured to separate the traffic generated based on router ID number.

Network Diagram

IP Allocation

  • AS 1: 192.168.0.0/16
    • CE 1: 192.168.0.1/32
    • CE 2: 192.168.0.2/32
    • Link 111: 192.168.1.0/31
    • Link 216: 192.168.1.2/31
  • AS 100: 10.10.0.0/16
    • PE 11: 10.10.0.11/32
    • P 12: 10.10.0.12/32
    • C-CE 13: 10.10.0.13/32
    • C-CE 14: 10.10.0.14/32
    • P 15: 10.10.0.15/32
    • PE 16: 10.10.0.16/32
    • Internal Links: 10.10.1.0/16
  • AS 200: 172.16.0.0/16
    • C-PE 21: 172.16.0.21/32
    • C-P 22: 172.16.0.22/32
    • C-PE 23: 172.16.0.23/32
    • Link 1321: 172.16.255.0/31
    • Link 1423: 172.16.255.2/31

Full Configuration

root@MX# show | no-more 
## Last changed: 2018-03-04 11:27:16 UTC
version 15.1F4.15;
groups {
    isis {
        logical-systems {
            <*> {
                protocols {
                    isis {
                        level 1 disable;
                        interface <*> {
                            point-to-point;
                        }
                    }
                }
            }
        }
    }
}
apply-groups isis;
system {
    host-name MX;
    root-authentication {
        encrypted-password "$5$L3F31155$kVyagZl2v/WM9s32/hi7VCXxM5o0vupYD.LO3uvCif4"; ## SECRET-DATA
    }
    services {
        ssh;
        netconf {
            ssh;
        }
    }
    syslog {
        user * {
            any emergency;
        }
        file messages {
            any notice;
            authorization info;
        }
        file interactive-commands {
            interactive-commands any;
        }
    }
}
logical-systems {
    1-CE {
        interfaces {
            ge-0/0/0 {
                unit 111 {
                    vlan-id 111;
                    family inet {
                        address 192.168.1.0/31;
                    }
                }
            }
            lo0 {
                unit 1 {
                    family inet {
                        address 192.168.0.1/32;
                    }
                }
            }
        }
        protocols {
            bgp {
                group AS100 {
                    type external;
                    export to-bgp;
                    peer-as 100;
                    neighbor 192.168.1.1;
                }
            }
        }
        policy-options {
            policy-statement to-bgp {
                from protocol direct;
                then accept;
            }
        }
        routing-options {
            autonomous-system 1;
        }
    }
    11-PE {
        interfaces {
            ge-0/0/0 {
                unit 1112 {
                    vlan-id 1112;
                    family inet {
                        address 10.10.1.0/31;
                    }
                    family iso;
                }
            }
            ge-0/0/1 {
                unit 111 {
                    vlan-id 111;
                    family inet {
                        address 192.168.1.1/31;
                    }
                }
            }
            lo0 {
                unit 11 {
                    family inet {
                        address 10.10.0.11/32;
                    }
                    family iso {
                        address 49.0100.0100.1000.0011.00;
                    }
                }
            }
        }
        protocols {
            bgp {
                group internal {
                    type internal;
                    local-address 10.10.0.11;
                    export to-ibgp;
                    neighbor 10.10.0.12;
                    neighbor 10.10.0.13;
                    neighbor 10.10.0.14;
                    neighbor 10.10.0.15;
                    neighbor 10.10.0.16;
                }
                group to-customer {
                    type external;
                    peer-as 1;
                    as-override;
                    neighbor 192.168.1.0;
                }
            }
            isis {
                interface ge-0/0/0.1112;
                interface lo0.11;
            }
        }
        policy-options {
            policy-statement to-ibgp {
                term nhs {
                    from protocol bgp;
                    then {
                        next-hop self;
                    }
                }
            }
        }
        routing-options {
            autonomous-system 100;
        }
    }
    12-P {
        interfaces {
            ge-0/0/0 {
                unit 1213 {
                    vlan-id 1213;
                    family inet {
                        address 10.10.1.2/31;
                    }
                    family iso;
                }
            }
            ge-0/0/1 {
                unit 1112 {
                    vlan-id 1112;
                    family inet {
                        address 10.10.1.1/31;
                    }
                    family iso;
                }
            }
            lo0 {
                unit 12 {
                    family inet {
                        address 10.10.0.12/32;
                    }
                    family iso {
                        address 49.0100.0100.1000.0012.00;
                    }
                }
            }
        }
        protocols {
            bgp {
                group internal {
                    type internal;
                    local-address 10.10.0.12;
                    neighbor 10.10.0.11;
                    neighbor 10.10.0.13;
                    neighbor 10.10.0.14;
                    neighbor 10.10.0.15;
                    neighbor 10.10.0.16;
                }
            }
            isis {
                interface all;
            }
        }
        routing-options {
            autonomous-system 100;
        }
    }
    13-C-CE {
        interfaces {
            ge-0/0/0 {
                unit 1321 {
                    vlan-id 1321;
                    family inet {
                        address 172.16.255.0/31;
                    }
                    family mpls;
                }
            }
            ge-0/0/1 {
                unit 1213 {
                    vlan-id 1213;
                    family inet {
                        address 10.10.1.3/31;
                    }
                    family iso;
                }
            }
            lo0 {
                unit 13 {
                    family inet {
                        address 10.10.0.13/32;
                    }
                    family iso {
                        address 49.0100.0100.1000.0013.00;
                    }
                }
            }
        }
        protocols {
            mpls {
                interface ge-0/0/0.1321;
            }
            bgp {
                group to-isp {
                    type external;
                    export internal;
                    peer-as 200;
                    neighbor 172.16.255.1 {
                        family inet {
                            labeled-unicast;
                        }
                    }
                }
                group internal {
                    type internal;
                    local-address 10.10.0.13;
                    export to-ibgp;
                    neighbor 10.10.0.11;
                    neighbor 10.10.0.12;
                    neighbor 10.10.0.14;
                    neighbor 10.10.0.15;
                    neighbor 10.10.0.16;
                }
            }
            isis {
                interface all;
            }
        }
        policy-options {
            policy-statement internal {
                term internal {
                    from protocol [ isis direct ];
                    then accept;
                }
                term reject {
                    then reject;
                }
            }
            policy-statement to-ibgp {
                term nhs {
                    from protocol bgp;
                    then {
                        next-hop self;
                    }
                }
            }
        }
        routing-options {
            autonomous-system 100;
        }
    }
    14-C-CE {
        interfaces {
            ge-0/0/0 {
                unit 1415 {
                    vlan-id 1415;
                    family inet {
                        address 10.10.1.4/31;
                    }
                    family iso;
                }
                unit 1423 {
                    vlan-id 1423;
                    family inet {
                        address 172.16.255.2/31;
                    }
                    family mpls;
                }
            }
            lo0 {
                unit 14 {
                    family inet {
                        address 10.10.0.14/32;
                    }
                    family iso {
                        address 49.0100.0100.1000.0014.00;
                    }
                }
            }
        }
        protocols {
            mpls {
                interface ge-0/0/0.1423;
            }
            bgp {
                group to-isp {
                    type external;
                    export internal;
                    peer-as 200;
                    neighbor 172.16.255.3 {
                        family inet {
                            labeled-unicast;
                        }
                    }
                }
                group internal {
                    type internal;
                    local-address 10.10.0.14;
                    export to-ibgp;
                    neighbor 10.10.0.11;
                    neighbor 10.10.0.12;
                    neighbor 10.10.0.13;
                    neighbor 10.10.0.15;
                    neighbor 10.10.0.16;
                }
            }
            isis {
                interface all;
            }
        }
        policy-options {
            policy-statement internal {
                term internal {
                    from protocol [ isis direct ];
                    then accept;
                }
                term reject {
                    then reject;
                }
            }
            policy-statement to-ibgp {
                term nhs {
                    from protocol bgp;
                    then {
                        next-hop self;
                    }
                }
            }
        }
        routing-options {
            autonomous-system 100;
        }
    }
    15-P {
        interfaces {
            ge-0/0/0 {
                unit 1516 {
                    vlan-id 1516;
                    family inet {
                        address 10.10.1.6/31;
                    }
                    family iso;
                }
            }
            ge-0/0/1 {
                unit 1415 {
                    vlan-id 1415;
                    family inet {
                        address 10.10.1.5/31;
                    }
                    family iso;
                }
            }
            lo0 {
                unit 15 {
                    family inet {
                        address 10.10.0.15/32;
                    }
                    family iso {
                        address 49.0100.0100.1000.0015.00;
                    }
                }
            }
        }
        protocols {
            bgp {
                group internal {
                    type internal;
                    local-address 10.10.0.15;
                    neighbor 10.10.0.11;
                    neighbor 10.10.0.12;
                    neighbor 10.10.0.13;
                    neighbor 10.10.0.14;
                    neighbor 10.10.0.16;
                }
            }
            isis {
                interface all;
            }
        }
        routing-options {
            autonomous-system 100;
        }
    }
    16-PE {
        interfaces {
            ge-0/0/1 {
                unit 216 {
                    vlan-id 216;
                    family inet {
                        address 192.168.1.3/31;
                    }
                }
                unit 1516 {
                    vlan-id 1516;
                    family inet {
                        address 10.10.1.7/31;
                    }
                    family iso;
                }
            }
            lo0 {
                unit 16 {
                    family inet {
                        address 10.10.0.16/32;
                    }
                    family iso {
                        address 49.0100.0100.1000.0016.00;
                    }
                }
            }
        }
        protocols {
            bgp {
                group internal {
                    type internal;
                    local-address 10.10.0.16;
                    export to-ibgp;
                    neighbor 10.10.0.11;
                    neighbor 10.10.0.12;
                    neighbor 10.10.0.13;
                    neighbor 10.10.0.14;
                    neighbor 10.10.0.15;
                }
                group to-customer {
                    type external;
                    peer-as 1;
                    as-override;
                    neighbor 192.168.1.2;
                }
            }
            isis {
                interface ge-0/0/1.1516;
                interface lo0.16;
            }
        }
        policy-options {
            policy-statement to-ibgp {
                term nhs {
                    from protocol bgp;
                    then {
                        next-hop self;
                    }
                }
            }
        }
        routing-options {
            autonomous-system 100;
        }
    }
    2-CE {
        interfaces {
            ge-0/0/0 {
                unit 216 {
                    vlan-id 216;
                    family inet {
                        address 192.168.1.2/31;
                    }
                }
            }
            lo0 {
                unit 2 {
                    family inet {
                        address 192.168.0.2/32;
                    }
                }
            }
        }
        protocols {
            bgp {
                group AS100 {
                    type external;
                    export to-bgp;
                    peer-as 100;
                    neighbor 192.168.1.3;
                }
            }
        }
        policy-options {
            policy-statement to-bgp {
                from protocol direct;
                then accept;
            }
        }
        routing-options {
            autonomous-system 1;
        }
    }
    21-C-PE {
        interfaces {
            ge-0/0/0 {
                unit 2122 {
                    vlan-id 2122;
                    family inet {
                        address 172.16.1.0/31;
                    }
                    family iso;
                    family mpls;
                }
            }
            ge-0/0/1 {
                unit 1321 {
                    vlan-id 1321;
                    family inet {
                        address 172.16.255.1/31;
                    }
                    family mpls;
                }
            }
            lo0 {
                unit 21 {
                    family inet {
                        address 172.16.0.21/32;
                    }
                    family iso {
                        address 49.0200.1720.1600.0021.00;
                    }
                }
            }
        }
        protocols {
            mpls {
                interface all;
            }
            bgp {
                group pe-pe {
                    type internal;
                    local-address 172.16.0.21;
                    neighbor 172.16.0.23 {
                        family inet-vpn {
                            any;
                        }
                    }
                }
            }
            isis {
                interface all;
            }
            ldp {
                interface ge-0/0/0.2122;
                interface lo0.21;
            }
        }
        routing-instances {
            vpn-isp {
                instance-type vrf;
                interface ge-0/0/1.1321;
                route-distinguisher 172.16.0.21:100;
                vrf-target target:200:100;
                protocols {
                    bgp {
                        group to-isp {
                            peer-as 100;
                            neighbor 172.16.255.0 {
                                family inet {
                                    labeled-unicast;
                                }
                                as-override;
                            }
                        }
                    }
                    mpls {
                        interface all;
                    }
                }
            }
        }
        routing-options {
            autonomous-system 200;
        }
    }
    22-C-P {
        interfaces {
            ge-0/0/0 {
                unit 2223 {
                    vlan-id 2223;
                    family inet {
                        address 172.16.1.2/31;
                    }
                    family iso;
                    family mpls;
                }
            }
            ge-0/0/1 {
                unit 2122 {
                    vlan-id 2122;
                    family inet {
                        address 172.16.1.1/31;
                    }
                    family iso;
                    family mpls;
                }
            }
            lo0 {
                unit 22 {
                    family inet {
                        address 172.16.0.22/32;
                    }
                    family iso {
                        address 49.0200.1720.1600.0022.00;
                    }
                }
            }
        }
        protocols {
            mpls {
                interface all;
            }
            isis {
                interface all;
            }
            ldp {
                interface ge-0/0/0.2223;
                interface ge-0/0/1.2122;
                interface lo0.22;
            }
        }
    }
    23-C-PE {
        interfaces {
            ge-0/0/1 {
                unit 1423 {
                    vlan-id 1423;
                    family inet {
                        address 172.16.255.3/31;
                    }
                    family mpls;
                }
                unit 2223 {
                    vlan-id 2223;
                    family inet {
                        address 172.16.1.3/31;
                    }
                    family iso;
                    family mpls;
                }
            }
            lo0 {
                unit 23 {
                    family inet {
                        address 172.16.0.23/32;
                    }
                    family iso {
                        address 49.0200.1720.1600.0023.00;
                    }
                }
            }
        }
        protocols {
            mpls {
                interface all;
            }
            bgp {
                group pe-pe {
                    type internal;
                    local-address 172.16.0.23;
                    neighbor 172.16.0.21 {
                        family inet-vpn {
                            any;
                        }
                    }
                }
            }
            isis {
                interface all;
            }
            ldp {
                interface ge-0/0/1.2223;
                interface lo0.23;
            }
        }
        routing-instances {
            vpn-isp {
                instance-type vrf;
                interface ge-0/0/1.1423;
                route-distinguisher 172.16.0.23:100;
                vrf-target target:200:100;
                protocols {
                    bgp {
                        group to-isp {
                            peer-as 100;
                            neighbor 172.16.255.2 {
                                family inet {
                                    labeled-unicast;
                                }
                                as-override;
                            }
                        }
                    }
                    mpls {
                        interface all;
                    }
                }
            }
        }
        routing-options {
            autonomous-system 200;
        }
    }
}
interfaces {
    ge-0/0/0 {
        description "to ge-0/0/1";
        vlan-tagging;
    }
    ge-0/0/1 {
        description "to ge-0/0/0";
        vlan-tagging;
    }
    ge-0/0/2 {
        description "to ge-0/0/3";
        vlan-tagging;
    }
    ge-0/0/3 {
        description "to ge-0/0/2";
        vlan-tagging;
    }
    ge-0/0/4 {
        description "to ge-0/0/5";
        vlan-tagging;
    }
    ge-0/0/5 {
        description "to ge-0/0/4";
        vlan-tagging;
    }
    ge-0/0/6 {
        description "to ge-0/0/7";
        vlan-tagging;
    }
    ge-0/0/7 {
        description "to ge-0/0/6";
        vlan-tagging;
    }
    ge-0/0/8 {
        description "to eth1";
    }
    ge-0/0/9 {
        description "to eth2";
    }
    fxp0 {
        unit 0 {
            family inet {
                address 192.168.83.11/24;
            }
        }
    }
}

Verification

root@MX:1-CE> show route                             

inet.0: 9 destinations, 9 routes (9 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

10.10.0.14/32      *[BGP/170] 00:17:04, localpref 100
                      AS path: 100 200 200 I, validation-state: unverified
                    > to 192.168.1.1 via ge-0/0/0.111
10.10.0.15/32      *[BGP/170] 00:17:04, localpref 100
                      AS path: 100 200 200 I, validation-state: unverified
                    > to 192.168.1.1 via ge-0/0/0.111
10.10.0.16/32      *[BGP/170] 00:17:04, localpref 100
                      AS path: 100 200 200 I, validation-state: unverified
                    > to 192.168.1.1 via ge-0/0/0.111
10.10.1.4/31       *[BGP/170] 00:17:04, localpref 100
                      AS path: 100 200 200 I, validation-state: unverified
                    > to 192.168.1.1 via ge-0/0/0.111
10.10.1.6/31       *[BGP/170] 00:17:04, localpref 100
                      AS path: 100 200 200 I, validation-state: unverified
                    > to 192.168.1.1 via ge-0/0/0.111
192.168.0.1/32     *[Direct/0] 03:17:20
                    > via lo0.1
192.168.0.2/32     *[BGP/170] 00:08:30, localpref 100
                      AS path: 100 100 I, validation-state: unverified
                    > to 192.168.1.1 via ge-0/0/0.111
192.168.1.0/31     *[Direct/0] 03:16:25
                    > via ge-0/0/0.111
192.168.1.0/32     *[Local/0] 03:16:25
                      Local via ge-0/0/0.111

root@MX:1-CE> ping 192.168.0.2 source 192.168.0.1 count 3 
PING 192.168.0.2 (192.168.0.2): 56 data bytes
64 bytes from 192.168.0.2: icmp_seq=0 ttl=46 time=12.347 ms
64 bytes from 192.168.0.2: icmp_seq=1 ttl=46 time=10.568 ms
64 bytes from 192.168.0.2: icmp_seq=2 ttl=46 time=7.953 ms

--- 192.168.0.2 ping statistics ---
3 packets transmitted, 3 packets received, 0% packet loss
round-trip min/avg/max/stddev = 7.953/10.289/12.347/1.805 ms

Label Operation

1. From router 1-CE to 13-C-CE will flow simple traffic.

root@MX:11-PE> show route 192.168.0.2 

inet.0: 15 destinations, 21 routes (15 active, 0 holddown, 1 hidden)
+ = Active Route, - = Last Active, * = Both

192.168.0.2/32     *[BGP/170] 00:10:57, localpref 100, from 10.10.0.16
                      AS path: 1 I, validation-state: unverified
                    > to 10.10.1.1 via ge-0/0/0.1112

2. Router 13-C-CE will push BGP-LU label.

root@MX:13-C-CE> show route 192.168.0.2 

inet.0: 15 destinations, 20 routes (15 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

192.168.0.2/32     *[BGP/170] 00:11:33, localpref 100, from 10.10.0.16
                      AS path: 1 I, validation-state: unverified
                    > to 172.16.255.1 via ge-0/0/0.1321, Push 299952

3. Router 21-C-PE will swap BGP-LU label and will push an LDP label.

root@MX:21-C-PE> show route 192.168.0.2 

root@MX:21-C-PE> show route label 299952 

vpn-isp.mpls.0: 6 destinations, 6 routes (6 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

299952             *[VPN/170] 00:35:44, metric2 1, from 172.16.0.23
                    > to 172.16.1.1 via ge-0/0/0.2122, Swap 299952, Push 299792(top)

root@MX:21-C-PE> show route label 299792    

mpls.0: 10 destinations, 10 routes (10 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

299792             *[LDP/9] 03:22:12, metric 1
                    > to 172.16.1.1 via ge-0/0/0.2122, Swap 299792

root@MX:21-C-PE> show route 172.16.0.23/32 protocol ldp 

inet.0: 7 destinations, 7 routes (7 active, 0 holddown, 0 hidden)

inet.3: 2 destinations, 2 routes (2 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

172.16.0.23/32     *[LDP/9] 03:24:05, metric 1
                    > to 172.16.1.1 via ge-0/0/0.2122, Push 299792

4. Router 22-C-P is PHP router and will pop the LDP label.

root@MX:22-C-P> show route label 299792 

mpls.0: 8 destinations, 8 routes (8 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

299792             *[LDP/9] 03:24:49, metric 1
                    > to 172.16.1.3 via ge-0/0/0.2223, Pop      
299792(S=0)        *[LDP/9] 03:24:49, metric 1
                    > to 172.16.1.3 via ge-0/0/0.2223, Pop   

5. Router 23-C-PE will swap BGP-LU label.

root@MX:23-C-PE> show route label 299952  

mpls.0: 10 destinations, 10 routes (10 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

299952             *[VPN/170] 00:42:46
                    > to 172.16.255.2 via ge-0/0/1.1423, Swap 299936

6. Router 14-C-CE will pop the BGP-LU label and will forward simple traffic.

root@MX:14-C-CE> show route label 299936 

mpls.0: 6 destinations, 6 routes (6 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

299936             *[VPN/170] 00:44:10
                    > to 10.10.1.5 via ge-0/0/0.1415, Pop      
299936(S=0)        *[VPN/170] 00:44:10
                    > to 10.10.1.5 via ge-0/0/0.1415, Pop      

root@MX:14-C-CE> show route 192.168.0.2 

inet.0: 15 destinations, 20 routes (15 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

192.168.0.2/32     *[BGP/170] 00:21:54, localpref 100, from 10.10.0.16
                      AS path: 1 I, validation-state: unverified
                    > to 10.10.1.5 via ge-0/0/0.1415

Documentation

Carrier-of-Carriers with VPN Service Provider as the Customer on a vMX

Implementation description

In this article, I will build a Carrier-of-Carriers with VPN Service Provider as the customer setup on a single vMX. Each individual router will be configured as a logical router. The vMX router has a back-to-back connection between ge-0/0/0 and ge-0/0/1 ports. For each different link, a different VLAN number will be configured to separate the traffic generated based on router ID number.

Network Diagram

IP Allocation

  • AS 1: 192.168.0.0/16
    • CE 1: 192.168.0.1/32
    • CE 2: 192.168.0.2/32
    • Link 111: 192.168.1.0/31
    • Link 216: 192.168.1.2/31
  • AS 100: 10.10.0.0/16
    • PE 11: 10.10.0.11/32
    • P 12: 10.10.0.12/32
    • C-CE 13: 10.10.0.13/32
    • C-CE 14: 10.10.0.14/32
    • P 15: 10.10.0.15/32
    • PE 16: 10.10.0.16/32
    • Internal Links: 10.10.1.0/16
  • AS 200: 172.16.0.0/16
    • C-PE 21: 172.16.0.21/32
    • C-P 22: 172.16.0.22/32
    • C-PE 23: 172.16.0.23/32
    • Link 1321: 172.16.255.0/31
    • Link 1423: 172.16.255.2/31

Full Configuration

root@MX# show | no-more 
## Last changed: 2017-12-27 03:07:53 UTC
version 15.1F4.15;
groups {
    isis {
        logical-systems {
            <*> {
                protocols {
                    isis {
                        level 1 disable;
                        interface <*> {
                            point-to-point;
                        }
                    }
                }
            }
        }
    }
}
apply-groups isis;
system {
    host-name MX;
    root-authentication {
        encrypted-password "$5$L3F31155$kVyagZl2v/WM9s32/hi7VCXxM5o0vupYD.LO3uvCif4"; ## SECRET-DATA
    }
    services {
        ssh;
        netconf {
            ssh;
        }
    }
    syslog {
        user * {
            any emergency;
        }
        file messages {
            any notice;
            authorization info;
        }
        file interactive-commands {
            interactive-commands any;
        }
    }
}
logical-systems {
    1-CE {
        interfaces {
            ge-0/0/0 {
                unit 111 {
                    vlan-id 111;
                    family inet {
                        address 192.168.1.0/31;
                    }
                }
            }
            lo0 {
                unit 1 {
                    family inet {
                        address 192.168.0.1/32;
                    }
                }
            }
        }
        protocols {
            bgp {
                group AS100 {
                    type external;
                    export to-bgp;
                    peer-as 100;
                    neighbor 192.168.1.1;
                }
            }
        }
        policy-options {
            policy-statement to-bgp {
                from protocol direct;
                then accept;
            }
        }
        routing-options {
            autonomous-system 1;
        }
    }
    11-PE {
        interfaces {
            ge-0/0/0 {
                unit 1112 {
                    vlan-id 1112;
                    family inet {
                        address 10.10.1.0/31;
                    }
                    family iso;
                    family mpls;
                }
            }
            ge-0/0/1 {
                unit 111 {
                    vlan-id 111;
                    family inet {
                        address 192.168.1.1/31;
                    }
                }
            }
            lo0 {
                unit 11 {
                    family inet {
                        address 10.10.0.11/32;
                    }
                    family iso {
                        address 49.0100.0100.1000.0011.00;
                    }
                }
            }
        }
        protocols {
            mpls {
                interface all;
            }
            bgp {
                group internal {
                    type internal;
                    local-address 10.10.0.11;
                    neighbor 10.10.0.13 {
                        family inet {
                            labeled-unicast {
                                resolve-vpn;
                            }
                        }
                    }
                    neighbor 10.10.0.16 {
                        family inet-vpn {
                            any;
                        }
                    }
                }
            }
            isis {
                interface ge-0/0/0.1112;
                interface lo0.11;
            }
            ldp {
                interface ge-0/0/0.1112;
                interface lo0.11;
            }
        }
        routing-instances {
            VPN {
                instance-type vrf;
                interface ge-0/0/1.111;
                route-distinguisher 10.10.0.11:1;
                vrf-target target:100:1;
                vrf-table-label;
                protocols {
                    bgp {
                        group vpn {
                            peer-as 1;
                            as-override;
                            neighbor 192.168.1.0;
                        }
                    }
                }
            }
        }
        routing-options {
            autonomous-system 100;
        }
    }
    12-P {
        interfaces {
            ge-0/0/0 {
                unit 1213 {
                    vlan-id 1213;
                    family inet {
                        address 10.10.1.2/31;
                    }
                    family iso;
                    family mpls;
                }
            }
            ge-0/0/1 {
                unit 1112 {
                    vlan-id 1112;
                    family inet {
                        address 10.10.1.1/31;
                    }
                    family iso;
                    family mpls;
                }
            }
            lo0 {
                unit 12 {
                    family inet {
                        address 10.10.0.12/32;
                    }
                    family iso {
                        address 49.0100.0100.1000.0012.00;
                    }
                }
            }
        }
        protocols {
            mpls {
                traffic-engineering {
                    bgp-igp;
                }
                interface all;
            }
            isis {
                interface all;
            }
            ldp {
                interface ge-0/0/0.1213;
                interface ge-0/0/1.1112;
                interface lo0.12;
            }
        }
    }
    13-C-CE {
        interfaces {
            ge-0/0/0 {
                unit 1321 {
                    vlan-id 1321;
                    family inet {
                        address 172.16.255.0/31;
                    }
                    family mpls;
                }
            }
            ge-0/0/1 {
                unit 1213 {
                    vlan-id 1213;
                    family inet {
                        address 10.10.1.3/31;
                    }
                    family iso;
                    family mpls;
                }
            }
            lo0 {
                unit 13 {
                    family inet {
                        address 10.10.0.13/32;
                    }
                    family iso {
                        address 49.0100.0100.1000.0013.00;
                    }
                }
            }
        }
        protocols {
            mpls {
                traffic-engineering {
                    bgp-igp;
                }
                interface all;
            }
            bgp {
                group internal {
                    type internal;
                    local-address 10.10.0.13;
                    neighbor 10.10.0.11 {
                        family inet {
                            labeled-unicast;
                        }
                    }
                }
                group to-isp {
                    type external;
                    export internal;
                    peer-as 200;
                    neighbor 172.16.255.1 {
                        family inet {
                            labeled-unicast;
                        }
                    }
                }
            }
            isis {
                interface all;
            }
            ldp {
                interface ge-0/0/1.1213;
                interface lo0.13;
            }
        }
        policy-options {
            policy-statement internal {
                term internal {
                    from protocol [ isis direct ldp ];
                    then accept;
                }
                term reject {
                    then reject;
                }
            }
        }
        routing-options {
            autonomous-system 100;
        }
    }
    14-C-CE {
        interfaces {
            ge-0/0/0 {
                unit 1415 {
                    vlan-id 1415;
                    family inet {
                        address 10.10.1.4/31;
                    }
                    family iso;
                    family mpls;
                }
                unit 1423 {
                    vlan-id 1423;
                    family inet {
                        address 172.16.255.2/31;
                    }
                    family mpls;
                }
            }
            lo0 {
                unit 14 {
                    family inet {
                        address 10.10.0.14/32;
                    }
                    family iso {
                        address 49.0100.0100.1000.0014.00;
                    }
                }
            }
        }
        protocols {
            mpls {
                traffic-engineering {
                    bgp-igp;
                }
                interface all;
            }
            bgp {
                group internal {
                    type internal;
                    local-address 10.10.0.14;
                    neighbor 10.10.0.16 {
                        family inet {
                            labeled-unicast;
                        }
                    }
                }
                group to-isp {
                    type external;
                    export internal;
                    peer-as 200;
                    neighbor 172.16.255.3 {
                        family inet {
                            labeled-unicast;
                        }
                    }
                }
            }
            isis {
                interface all;
            }
            ldp {
                interface ge-0/0/0.1415;
                interface lo0.14;
            }
        }
        policy-options {
            policy-statement internal {
                term internal {
                    from protocol [ isis direct ldp ];
                    then accept;
                }
                term reject {
                    then reject;
                }
            }
        }
        routing-options {
            autonomous-system 100;
        }
    }
    15-P {
        interfaces {
            ge-0/0/0 {
                unit 1516 {
                    vlan-id 1516;
                    family inet {
                        address 10.10.1.6/31;
                    }
                    family iso;
                    family mpls;
                }
            }
            ge-0/0/1 {
                unit 1415 {
                    vlan-id 1415;
                    family inet {
                        address 10.10.1.5/31;
                    }
                    family iso;
                    family mpls;
                }
            }
            lo0 {
                unit 15 {
                    family inet {
                        address 10.10.0.15/32;
                    }
                    family iso {
                        address 49.0100.0100.1000.0015.00;
                    }
                }
            }
        }
        protocols {
            mpls {
                traffic-engineering {
                    bgp-igp;
                }
                interface all;
            }
            isis {
                interface all;
            }
            ldp {
                interface ge-0/0/0.1516;
                interface ge-0/0/1.1415;
                interface lo0.15;
            }
        }
    }
    16-PE {
        interfaces {
            ge-0/0/1 {
                unit 216 {
                    vlan-id 216;
                    family inet {
                        address 192.168.1.3/31;
                    }
                }
                unit 1516 {
                    vlan-id 1516;
                    family inet {
                        address 10.10.1.7/31;
                    }
                    family iso;
                    family mpls;
                }
            }
            lo0 {
                unit 16 {
                    family inet {
                        address 10.10.0.16/32;
                    }
                    family iso {
                        address 49.0100.0100.1000.0016.00;
                    }
                }
            }
        }
        protocols {
            mpls {
                interface all;
            }
            bgp {
                group internal {
                    type internal;
                    local-address 10.10.0.16;
                    neighbor 10.10.0.11 {
                        family inet-vpn {
                            any;
                        }
                    }
                    neighbor 10.10.0.14 {
                        family inet {
                            labeled-unicast {
                                resolve-vpn;
                            }
                        }
                    }
                }
            }
            isis {
                interface ge-0/0/1.1516;
                interface lo0.16;
            }
            ldp {
                interface ge-0/0/1.1516;
                interface lo0.16;
            }
        }
        routing-instances {
            VPN {
                instance-type vrf;
                interface ge-0/0/1.216;
                route-distinguisher 10.10.0.16:1;
                vrf-target target:100:1;
                vrf-table-label;
                protocols {
                    bgp {
                        group vpn {
                            peer-as 1;
                            as-override;
                            neighbor 192.168.1.2;
                        }
                    }
                }
            }
        }
        routing-options {
            autonomous-system 100;
        }
    }
    2-CE {
        interfaces {
            ge-0/0/0 {
                unit 216 {
                    vlan-id 216;
                    family inet {
                        address 192.168.1.2/31;
                    }
                }
            }
            lo0 {
                unit 2 {
                    family inet {
                        address 192.168.0.2/32;
                    }
                }
            }
        }
        protocols {
            bgp {
                group AS100 {
                    type external;
                    export to-bgp;
                    peer-as 100;
                    neighbor 192.168.1.3;
                }
            }
        }
        policy-options {
            policy-statement to-bgp {
                from protocol direct;
                then accept;
            }
        }
        routing-options {
            autonomous-system 1;
        }
    }
    21-C-PE {
        interfaces {
            ge-0/0/0 {
                unit 2122 {
                    vlan-id 2122;
                    family inet {
                        address 172.16.1.0/31;
                    }
                    family iso;
                    family mpls;
                }
            }
            ge-0/0/1 {
                unit 1321 {
                    vlan-id 1321;
                    family inet {
                        address 172.16.255.1/31;
                    }
                    family mpls;
                }
            }
            lo0 {
                unit 21 {
                    family inet {
                        address 172.16.0.21/32;
                    }
                    family iso {
                        address 49.0200.1720.1600.0021.00;
                    }
                }
            }
        }
        protocols {
            mpls {
                interface all;
            }
            bgp {
                group pe-pe {
                    type internal;
                    local-address 172.16.0.21;
                    neighbor 172.16.0.23 {
                        family inet-vpn {
                            any;
                        }
                    }
                }
            }
            isis {
                interface all;
            }
            ldp {
                interface ge-0/0/0.2122;
                interface lo0.21;
            }
        }
        routing-instances {
            vpn-isp {
                instance-type vrf;
                interface ge-0/0/1.1321;
                route-distinguisher 172.16.0.21:100;
                vrf-target target:200:100;
                protocols {
                    bgp {
                        group to-isp {
                            peer-as 100;
                            neighbor 172.16.255.0 {
                                family inet {
                                    labeled-unicast;
                                }
                                as-override;
                            }
                        }
                    }
                    mpls {
                        interface all;
                    }
                }
            }
        }
        routing-options {
            autonomous-system 200;
        }
    }
    22-C-P {
        interfaces {
            ge-0/0/0 {
                unit 2223 {
                    vlan-id 2223;
                    family inet {
                        address 172.16.1.2/31;
                    }
                    family iso;
                    family mpls;
                }
            }
            ge-0/0/1 {
                unit 2122 {
                    vlan-id 2122;
                    family inet {
                        address 172.16.1.1/31;
                    }
                    family iso;
                    family mpls;
                }
            }
            lo0 {
                unit 22 {
                    family inet {
                        address 172.16.0.22/32;
                    }
                    family iso {
                        address 49.0200.1720.1600.0022.00;
                    }
                }
            }
        }
        protocols {
            mpls {
                interface all;
            }
            isis {
                interface all;
            }
            ldp {
                interface ge-0/0/0.2223;
                interface ge-0/0/1.2122;
                interface lo0.22;
            }
        }
    }
    23-C-PE {
        interfaces {
            ge-0/0/1 {
                unit 1423 {
                    vlan-id 1423;
                    family inet {
                        address 172.16.255.3/31;
                    }
                    family mpls;
                }
                unit 2223 {
                    vlan-id 2223;
                    family inet {
                        address 172.16.1.3/31;
                    }
                    family iso;
                    family mpls;
                }
            }
            lo0 {
                unit 23 {
                    family inet {
                        address 172.16.0.23/32;
                    }
                    family iso {
                        address 49.0200.1720.1600.0023.00;
                    }
                }
            }
        }
        protocols {
            mpls {
                interface all;
            }
            bgp {
                group pe-pe {
                    type internal;
                    local-address 172.16.0.23;
                    neighbor 172.16.0.21 {
                        family inet-vpn {
                            any;
                        }
                    }
                }
            }
            isis {
                interface all;
            }
            ldp {
                interface ge-0/0/1.2223;
                interface lo0.23;
            }
        }
        routing-instances {
            vpn-isp {
                instance-type vrf;
                interface ge-0/0/1.1423;
                route-distinguisher 172.16.0.23:100;
                vrf-target target:200:100;
                protocols {
                    bgp {
                        group to-isp {
                            peer-as 100;
                            neighbor 172.16.255.2 {
                                family inet {
                                    labeled-unicast;
                                }
                                as-override;
                            }
                        }
                    }
                    mpls {
                        interface all;
                    }
                }
            }
        }
        routing-options {
            autonomous-system 200;
        }
    }
}
interfaces {
    ge-0/0/0 {
        description "to ge-0/0/1";
        vlan-tagging;
    }
    ge-0/0/1 {
        description "to ge-0/0/0";
        vlan-tagging;
    }
    ge-0/0/2 {
        description "to ge-0/0/3";
        vlan-tagging;
    }
    ge-0/0/3 {
        description "to ge-0/0/2";
        vlan-tagging;
    }
    ge-0/0/4 {
        description "to ge-0/0/5";
        vlan-tagging;
    }
    ge-0/0/5 {
        description "to ge-0/0/4";
        vlan-tagging;
    }
    ge-0/0/6 {
        description "to ge-0/0/7";
        vlan-tagging;
    }
    ge-0/0/7 {
        description "to ge-0/0/6";
        vlan-tagging;
    }
    ge-0/0/8 {
        description "to eth1";
    }
    ge-0/0/9 {
        description "to eth2";
    }
    fxp0 {
        unit 0 {
            family inet {
                address 192.168.83.11/24;
            }
        }
    }
}

Verification

root@MX:1-CE> show route 

inet.0: 5 destinations, 5 routes (5 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

192.168.0.1/32     *[Direct/0] 07:13:24
                    > via lo0.1
192.168.0.2/32     *[BGP/170] 00:05:32, localpref 100
                      AS path: 100 100 I, validation-state: unverified
                    > to 192.168.1.1 via ge-0/0/0.111
192.168.1.0/31     *[Direct/0] 01:40:58
                    > via ge-0/0/0.111
192.168.1.0/32     *[Local/0] 01:40:59
                      Local via ge-0/0/0.111
192.168.1.2/31     *[BGP/170] 00:05:32, localpref 100
                      AS path: 100 I, validation-state: unverified
                    > to 192.168.1.1 via ge-0/0/0.111

root@MX:1-CE> ping 192.168.0.2 source 192.168.0.1 count 3 
PING 192.168.0.2 (192.168.0.2): 56 data bytes
64 bytes from 192.168.0.2: icmp_seq=0 ttl=46 time=21.322 ms
64 bytes from 192.168.0.2: icmp_seq=1 ttl=46 time=13.263 ms
64 bytes from 192.168.0.2: icmp_seq=2 ttl=46 time=14.254 ms

--- 192.168.0.2 ping statistics ---
3 packets transmitted, 3 packets received, 0% packet loss
round-trip min/avg/max/stddev = 13.263/16.280/21.322/3.588 ms

Label Operation

1. Router 1-CE will send simple traffic:

root@MX:1-CE> show route 192.168.0.2  

inet.0: 5 destinations, 5 routes (5 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

192.168.0.2/32     *[BGP/170] 00:29:05, localpref 100
                      AS path: 100 100 I, validation-state: unverified
                    > to 192.168.1.1 via ge-0/0/0.111

2. Router 11-PE will push L3VPN label, BGP-LU label and LDP trasport label:

root@MX:11-PE> show route 192.168.0.2 

VPN.inet.0: 5 destinations, 6 routes (5 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

192.168.0.2/32     *[BGP/170] 00:30:40, localpref 100, from 10.10.0.16
                      AS path: 1 I, validation-state: unverified
                    > to 10.10.1.1 via ge-0/0/0.1112, Push 16, Push 300112, Push 299856(top)

3. Router 12-P is PHP and will pop LDP label:

root@MX:12-P> show route table mpls.0 label 299856 

mpls.0: 8 destinations, 8 routes (8 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

299856             *[LDP/9] 01:22:43, metric 1
                    > to 10.10.1.3 via ge-0/0/0.1213, Pop      
299856(S=0)        *[LDP/9] 01:22:43, metric 1
                    > to 10.10.1.3 via ge-0/0/0.1213, Pop      

3. Router 13-C-CE is transit router for BGP-LU tunnel and will swap BGP label:

root@MX:13-C-CE> show route label 300112 

mpls.0: 16 destinations, 16 routes (16 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

300112             *[VPN/170] 00:36:20
                    > to 172.16.255.1 via ge-0/0/0.1321, Swap 300096

4. Router 21-C-PE will swap BGP label and push LDP label:

root@MX:21-C-PE> show route label 300096  

vpn-isp.mpls.0: 8 destinations, 8 routes (8 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

300096             *[VPN/170] 00:38:24, metric2 1, from 172.16.0.23
                    > to 172.16.1.1 via ge-0/0/0.2122, Swap 300080, Push 299840(top)

root@MX:21-C-PE> show route 192.168.0.2 

root@MX:21-C-PE> show route 10.10.0.16 

vpn-isp.inet.0: 12 destinations, 13 routes (12 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

10.10.0.16/32      *[BGP/170] 00:39:31, MED 1, localpref 100, from 172.16.0.23
                      AS path: 100 I, validation-state: unverified
                    > to 172.16.1.1 via ge-0/0/0.2122, Push 300080, Push 299840(top)

root@MX:21-C-PE> show route 172.16.0.23 

inet.0: 7 destinations, 7 routes (7 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

172.16.0.23/32     *[IS-IS/18] 02:17:08, metric 20
                    > to 172.16.1.1 via ge-0/0/0.2122

inet.3: 2 destinations, 2 routes (2 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

172.16.0.23/32     *[LDP/9] 02:17:08, metric 1
                    > to 172.16.1.1 via ge-0/0/0.2122, Push 299840

4. Router 22-C-P is PHP and will pop LDP label:

root@MX:22-C-P> show route label 299840 

mpls.0: 8 destinations, 8 routes (8 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

299840             *[LDP/9] 02:19:45, metric 1
                    > to 172.16.1.3 via ge-0/0/0.2223, Pop      
299840(S=0)        *[LDP/9] 02:19:45, metric 1
                    > to 172.16.1.3 via ge-0/0/0.2223, Pop      

5. Router 23-C-PE will swap BGP label:

root@MX:23-C-PE> show route 10.10.0.16 

vpn-isp.inet.0: 12 destinations, 13 routes (12 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

10.10.0.16/32      *[BGP/170] 00:45:36, MED 1, localpref 100
                      AS path: 100 I, validation-state: unverified
                    > to 172.16.255.2 via ge-0/0/1.1423, Push 300064

root@MX:23-C-PE> show route table mpls.0 label 300080 

mpls.0: 12 destinations, 12 routes (12 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

300080             *[VPN/170] 00:56:09
                    > to 172.16.255.2 via ge-0/0/1.1423, Swap 300064

6. Router 14-C-CE will swap LDP label:

root@MX:14-C-CE> show route label 300064 

mpls.0: 16 destinations, 16 routes (16 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

300064             *[VPN/170] 00:57:14
                    > to 10.10.1.5 via ge-0/0/0.1415, Swap 299872

root@MX:14-C-CE> show route 10.10.0.16 

inet.0: 14 destinations, 16 routes (14 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

10.10.0.16/32      *[LDP/9] 01:47:03, metric 1
                    > to 10.10.1.5 via ge-0/0/0.1415, Push 299872
                    [IS-IS/18] 01:47:03, metric 20
                    > to 10.10.1.5 via ge-0/0/0.1415

Note: Here BGP-LU is, in fact, internal LDP tunel. Only one transport label is needed.

7. Router 15-P is PHP router and will pop the LDP label:

root@MX:15-P> show route label 299872 

mpls.0: 8 destinations, 8 routes (8 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

299872             *[LDP/9] 01:56:46, metric 1
                    > to 10.10.1.7 via ge-0/0/0.1516, Pop      
299872(S=0)        *[LDP/9] 01:56:46, metric 1
                    > to 10.10.1.7 via ge-0/0/0.1516, Pop      

8. Router 16-PE will receive the traffic with only L3VPN label. It will pop the label and forward the traffic to de destination:

root@MX:16-PE> show route 192.168.0.2 

VPN.inet.0: 5 destinations, 6 routes (5 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

192.168.0.2/32     *[BGP/170] 02:45:28, localpref 100
                      AS path: 1 I, validation-state: unverified
                    > to 192.168.1.2 via ge-0/0/1.216

Documentation

Hardware Limitation and Planning for My MacBook Virtual Network

LAB Diagram and Devices BOM

My intention is to build and test the labs presented in MPLS in SDN Era Book:

As shown in the network diagram the BOM consist of:

  • 10 x Junos Logical Systems Devices: 2xH, 2xCE, 2xPE, 2xBR, 1xRR, 1xP
  • 4 x IOS XRv Virtual Routers Devices: 2xPE, 1xRR, 1xP

Note: H/CE/BR devices can be built also using simple instance-type virtual routers as no additional inside virtualization is needed.

  • 6 x Junos Routing Instances instance-type virtual routers: 2xH, 2xCE, 2xBR
  • 4 x Junos Logical Systems Devices: 2xPE, 1xRR, 1xP
  • 4 x IOS XRv Virtual Routers Devices: 2xPE, 1xRR, 1xP

Continue reading

vMX Initial Configuration

This article contains how I configure the vMX virtual router hosted in Ubuntu VM. Before starting I like to make a clear diagram into my head. 🙂

0. vMX Logical Network Diagram

Next table contains a summary of the external interfaces related to vMX. Internal back-to-back interfaces are not included in this table.

Network Connected Device Device Port Info IP
vmnet3 vMX – VCP fxp0 OoB MNG Net 192.168.83.11
vMX – VFP vfp-int OoB MNG Net 192.168.83.12
vmnet4 vMX – Logical Systems ge-0/0/8 Intelink with VLANs not configured
vMX – Ubuntu Server eth1 Intelink with VLANs 172.16.109.10
vmnet5 vMX – Logical Systems ge-0/0/9 Intelink with VLANs not configured
vMX – Ubuntu Server eth2 Intelink with VLANs 192.168.50.10

Note: vmnet4 and vmnet5 are the networks where I will connect vSRX and XRv devices.

1. Ubuntu Host VM Interfaces Configuration

I configure Ubuntu Host VM interfaces into my recent article Ubuntu VM Network Configuration. When I reboot using sudo reboot the VM and get into the system using ssh I get this output:

Welcome to Ubuntu 14.04.1 LTS (GNU/Linux 3.13.0-32-generic x86_64)

* Documentation: https://help.ubuntu.com/

System information as of Mon Nov 20 13:55:47 EET 2017

System load: 0.15 Processes: 407
 Usage of /: 23.4% of 48.11GB Users logged in: 0
 Memory usage: 1% IP address for eth0: 192.168.83.10
 Swap usage: 0% IP address for eth1: 172.16.109.10

Graph this data and manage this system at:
 https://landscape.canonical.com/

259 packages can be updated.
160 updates are security updates.

2. vMX VM Installation

I have installed vMX virtual router using Day One: vMX UP and Running book available on Juniper Networks website. The vMX VM deployment is out of the scope of this article.

3. HOST, VCP/vRE and VFP/vPFE Configuration

 silvia@vMX-ubuntu:~$ cd vmx-15.1F4-3/
 silvia@vMX-ubuntu:~/vmx-15.1F4-3$ cd config/
 silvia@vMX-ubuntu:~/vmx-15.1F4-3/config$ cat vmx.conf
 ##############################################################
 #
 # vmx.conf
 # Config file for vmx on the hypervisor.
 # Uses YAML syntax.
 # Leave a space after ":" to specify the parameter value.
 #
 ##############################################################

---
 #Configuration on the host side - management interface, VM images etc.
 HOST:
   identifier : vmx1 # Maximum 4 characters
   host-management-interface : eth0 <<-- Make sure here is the OoB MNG interface from Ubuntu VM
   routing-engine-image : "/home/silvia/vmx-15.1F4-3/images/jinstall64-vmx-15.1F4.15-domestic.img"
   routing-engine-hdd : "/home/silvia/vmx-15.1F4-3/images/vmxhdd.img"
   forwarding-engine-image : "/home/silvia/vmx-15.1F4-3/images/vFPC-20151203.img"

---
 #External bridge configuration
 BRIDGES:
   - type : external
     name : br-ext # Max 10 characters

---
 #vRE VM parameters
 CONTROL_PLANE:
   vcpus : 1
   memory-mb : 1024
   console_port: 8601

   interfaces :
      - type : static
        ipaddr : 192.168.83.11 <<-- Make sure here is the OoB MNG IP Add
        macaddr : "0A:00:DD:C0:DE:0E"

---
 #vPFE VM parameters
 FORWARDING_PLANE:
   memory-mb : 4096
   vcpus : 3
   console_port: 8602
   device-type : virtio

   interfaces :
     - type : static
       ipaddr : 192.168.83.12 <<-- Make sure here is the OoB MNG IP Add / same IP Subnet as vRE
       macaddr : "0A:00:DD:C0:DE:10"

---
 #Interfaces
 JUNOS_DEVICES: <<-- Maximum 10 are supported
   - interface : ge-0/0/0
     mac-address : "02:06:0A:0E:FF:F0"
     description : "ge-0/0/0 interface"

   - interface : ge-0/0/1
     mac-address : "02:06:0A:0E:FF:F1"
     description : "ge-0/0/1 interface"

   - interface : ge-0/0/2
     mac-address : "02:06:0A:0E:FF:F2"
     description : "ge-0/0/2 interface"

   - interface : ge-0/0/3
     mac-address : "02:06:0A:0E:FF:F3"
     description : "ge-0/0/3 interface"

   - interface : ge-0/0/4
     mac-address : "02:06:0A:0E:FF:F4"
     description : "ge-0/0/4 interface"

   - interface : ge-0/0/5
     mac-address : "02:06:0A:0E:FF:F5"
     description : "ge-0/0/5 interface"

   - interface : ge-0/0/6
     mac-address : "02:06:0A:0E:FF:F6"
     description : "ge-0/0/6 interface"

   - interface : ge-0/0/7
     mac-address : "02:06:0A:0E:FF:F7"
     description : "ge-0/0/7 interface"

   - interface : ge-0/0/8
     mac-address : "02:06:0A:0E:FF:F8"
     description : "ge-0/0/8 interface"

   - interface : ge-0/0/9
     mac-address : "02:06:0A:0E:FF:F9"
     description : "ge-0/0/9 interface"

4. Interfaces configuration (virtio)

Edit vmx-junosdev.conf file. I have used vi editor.

silvia@vMX-ubuntu:~$ cd vmx-15.1F4-3/config/
silvia@vMX-ubuntu:~/vmx-15.1F4-3/config$ vi vmx-junosdev.conf

Pres [i] to enter Insert mode. Edit the file according to the connectivity design. Pres [esc] to exit Insert mode. Then type :wq! to write and quit vi editor.

silvia@vMX-ubuntu:~/vmx-15.1F4-3/config$ cat vmx-junosdev.conf 
##############################################################
#
# vmx-junos-dev.conf
# - Config file for junos device bindings.
# - Uses YAML syntax. 
# - Leave a space after ":" to specify the parameter value.
# - For physical NIC, set the 'type' as 'host_dev'
# - For junos devices, set the 'type' as 'junos_dev' and
# set the mandatory parameter 'vm-name' to the name of
# the vPFE where the device exists
# - For bridge devices, set the 'type' as 'bridge_dev'
#
##############################################################
interfaces :

  - link_name : vmx_link1
    mtu : 1500
    endpoint_1 : 
      - type : junos_dev
        vm_name : vmx1 
        dev_name : ge-0/0/0
    endpoint_2 :
      - type : junos_dev
        vm_name : vmx1
        dev_name : ge-0/0/1

  - link_name : vmx_link2
    mtu : 1500
    endpoint_1 : 
      - type : junos_dev
        vm_name : vmx1
        dev_name : ge-0/0/2
    endpoint_2 :
      - type : junos_dev
        vm_name : vmx1
        dev_name : ge-0/0/3

  - link_name : vmx_link3
    endpoint_1 : 
      - type : junos_dev
        vm_name : vmx1
        dev_name : ge-0/0/4
    endpoint_2 :
      - type : junos_dev
        vm_name : vmx1
        dev_name : ge-0/0/5

  - link_name : vmx_link4
    endpoint_1 : 
      - type : junos_dev
        vm_name : vmx1
        dev_name : ge-0/0/6
    endpoint_2 :
      - type : junos_dev
        vm_name : vmx1
        dev_name : ge-0/0/7
 
  - link_name : vmx_link5
    endpoint_1 : 
      - type : junos_dev
        vm_name : vmx1
        dev_name : ge-0/0/8
    endpoint_2 :
      - type : host_dev
        dev_name : eth1

  - link_name : vmx_link6
    endpoint_1 :
      - type : junos_dev
        vm_name : vmx1
        dev_name : ge-0/0/9
    endpoint_2 :
      - type : host_dev
        dev_name : eth2

5. vMX RE-Installation

I will reinstall my vMX using the edited configuration files and the orchestration scrip vmx.sh.

silvia@vMX-ubuntu:~/vmx-15.1F4-3$ sudo ./vmx.sh -lv --install
[sudo] password for silvia: 
==================================================
 Welcome to VMX
==================================================
Date..............................................11/20/17 14:00:45
VMX Identifier....................................vmx1
Config file......................................./home/silvia/vmx-15.1F4-3/config/vmx.conf
Build Directory.................................../home/silvia/vmx-15.1F4-3/build/vmx1
Environment file................................../home/silvia/vmx-15.1F4-3/env/ubuntu_virtio.env
Junos Device Type.................................virtio
Initialize scripts................................[OK]
Copy images to build directory....................[OK]
==================================================
 VMX Environment Setup Completed
==================================================
==================================================
 VMX Install & Start
==================================================
Linux distribution................................ubuntu
Check GRUB........................................[Disabled]
Installation status of qemu-kvm...................[OK]
Installation status of libvirt-bin................[OK]
Installation status of bridge-utils...............[OK]
Installation status of python.....................[OK]
Installation status of libyaml-dev................[OK]
Installation status of python-yaml................[OK]
Installation status of numactl....................[OK]
Installation status of libnuma-dev................[OK]
Installation status of libparted0-dev.............[OK]
Installation status of libpciaccess-dev...........[OK]
Installation status of libyajl-dev................[OK]
Installation status of libxml2-dev................[OK]
Installation status of libglib2.0-dev.............[OK]
Installation status of libnl-dev..................[OK]
Check Kernel Version..............................[Disabled]
Check Qemu Version................................[Disabled]
Check libvirt Version.............................[Disabled]
Check virsh connectivity..........................[OK]
IXGBE Enabled.....................................[Disabled]
==================================================
 Pre-Install Checks Completed
==================================================
Check for VM vcp-vmx1.............................[Not Running]
Check for VM vfp-vmx1.............................[Not Running]
Cleanup VM states.................................[OK]
Check if bridge br-ext exists.....................[No]
Cleanup VM bridge br-ext..........................[OK]
Cleanup VM bridge br-int-vmx1.....................[OK]
==================================================
 VMX Stop Completed
==================================================
Check VCP image...................................[OK]
Check VFP image...................................[OK]
VMX Model.........................................FPC
Check VCP Config image............................[OK]
Check management interface........................[OK]
Setup huge pages to 8192..........................[OK]
Attempt to kill libvirt...........................[OK]
Attempt to start libvirt..........................[OK]
Sleep 2 secs......................................[OK]
Check libvirt support for hugepages...............[OK]
==================================================
 System Setup Completed
==================================================
Get Management Address of eth0....................[OK]
Generate libvirt files............................[OK]
Sleep 2 secs......................................[OK]
Find configured management interface..............eth0
Find existing management gateway..................eth0
Check if eth0 is already enslaved to br-ext.......[No]
Gateway interface needs change....................[Yes]
Create br-ext.....................................[OK]
Get Management Gateway............................192.168.83.1
Flush eth0........................................[OK]
Start br-ext......................................[OK]
Bind eth0 to br-ext...............................[OK]
Get Management MAC................................00:0c:29:9e:bb:5b
Assign Management MAC 00:0c:29:9e:bb:5b...........[OK]
Add default gw 192.168.83.1.......................[OK]
Create br-int-vmx1................................[OK]
Start br-int-vmx1.................................[OK]
Check and start default bridge....................[OK]
Define vcp-vmx1...................................[OK]
Define vfp-vmx1...................................[OK]
Wait 2 secs.......................................[OK]
Start vcp-vmx1....................................[OK]
Start vfp-vmx1....................................[OK]
Wait 2 secs.......................................[OK]
==================================================
 VMX Bringup Completed
==================================================
Check if br-ext is created........................[Created]
Check if br-int-vmx1 is created...................[Created]
Check if VM vcp-vmx1 is running...................[Running]
Check if VM vfp-vmx1 is running...................[Running]
Check if tap interface vcp_ext-vmx1 exists........[OK]
Check if tap interface vcp_int-vmx1 exists........[OK]
Check if tap interface vfp_ext-vmx1 exists........[OK]
Check if tap interface vfp_int-vmx1 exists........[OK]
==================================================
 VMX Status Verification Completed.
==================================================
Log file........................................../home/silvia/vmx-15.1F4-3/build/vmx1/logs/vmx_1511179244.log
==================================================
 Thankyou for using VMX
==================================================

6. Interfaces binding creation

First, let’s review what Linux bridges the vMX script just created when the vMX instance was deployed. This is done using the shell brctl show command:

silvia@vMX-ubuntu:~/vmx-15.1F4-3$ brctl show
bridge name bridge id STP enabled interfaces
br-ext 8000.000c299ebb5b   yes    br-ext-nic 
 ^-- the external bridge that is used for management of the vMX and the KVM host.                                 
                                  eth0
                                  vcp_ext-vmx1
                                  vfp_ext-vmx1
br-int-vmx1 8000.5254007e46ae yes br-int-vmx1-nic
 ^--the internal bridge that is used for communication between the VCP and VFP
                                  vcp_int-vmx1
                                  vfp_int-vmx1
virbr0 8000.52540007eaed   yes    ge-0.0.0-vmx1
                                  ge-0.0.1-vmx1
                                  ge-0.0.2-vmx1
                                  ge-0.0.3-vmx1
                                  ge-0.0.4-vmx1
                                  ge-0.0.5-vmx1
                                  ge-0.0.6-vmx1
                                  ge-0.0.7-vmx1
                                  ge-0.0.8-vmx1
                                  ge-0.0.9-vmx1
                                  virbr0-nic

Next, let’s check that the virtio binding has not been activated. Again I will use the orchestration script that Juniper provide with vMX:

silvia@vMX-ubuntu:~/vmx-15.1F4-3$ sudo ./vmx.sh --bind-check
Checking package ethtool..........................[OK]
Check Link vmx_link1(ge-0.0.0-vmx1, ge-0.0.1-vmx1)
[Not Present]
Check Link vmx_link2(ge-0.0.2-vmx1, ge-0.0.3-vmx1)
[Not Present]
Check Link vmx_link3(ge-0.0.4-vmx1, ge-0.0.5-vmx1)
[Not Present]
Check Link vmx_link4(ge-0.0.6-vmx1, ge-0.0.7-vmx1)
[Not Present]
Check Link vmx_link5(ge-0.0.8-vmx1, eth1).........[Not Present]
Check Link vmx_link6(ge-0.0.9-vmx1, eth2).........[Not Present]

Using –bind-dev option, I will create the binding:

silvia@vMX-ubuntu:~/vmx-15.1F4-3$ sudo ./vmx.sh --bind-dev
Checking package ethtool..........................[OK]
Bind Link vmx_link1(ge-0.0.0-vmx1, ge-0.0.1-vmx1)
[OK]
Bind Link vmx_link2(ge-0.0.2-vmx1, ge-0.0.3-vmx1)
[OK]
Bind Link vmx_link3(ge-0.0.4-vmx1, ge-0.0.5-vmx1)
[OK]
Bind Link vmx_link4(ge-0.0.6-vmx1, ge-0.0.7-vmx1)
[OK]
Bind Link vmx_link5(ge-0.0.8-vmx1, eth1)..........[OK]
Numa node for eth1................................-1
Cores servicing numa node -1......................
Pid of vfp-vmx1...................................8540
Pin vhost-8540 (PID=8552) to cores ...............taskset: failed to parse CPU list: 
[Failed]
Pin vhost-8540 (PID=8551) to cores ...............taskset: failed to parse CPU list: 
[Failed]
Pin vhost-8540 (PID=8550) to cores ...............taskset: failed to parse CPU list: 
[Failed]
Pin vhost-8540 (PID=8549) to cores ...............taskset: failed to parse CPU list: 
[Failed]
Pin vhost-8540 (PID=8548) to cores ...............taskset: failed to parse CPU list: 
[Failed]
Pin vhost-8540 (PID=8547) to cores ...............taskset: failed to parse CPU list: 
[Failed]
Pin vhost-8540 (PID=8546) to cores ...............taskset: failed to parse CPU list: 
[Failed]
Pin vhost-8540 (PID=8545) to cores ...............taskset: failed to parse CPU list: 
[Failed]
Pin vhost-8540 (PID=8544) to cores ...............taskset: failed to parse CPU list: 
[Failed]
Pin vhost-8540 (PID=8543) to cores ...............taskset: failed to parse CPU list: 
[Failed]
Pin vhost-8540 (PID=8542) to cores ...............taskset: failed to parse CPU list: 
[Failed]
Pin vhost-8540 (PID=8541) to cores ...............taskset: failed to parse CPU list: 
[Failed]
Bind Link vmx_link6(ge-0.0.9-vmx1, eth2)..........[OK]

^– The taskset command is used to achieve better performance in virtio mode.

I chose to ignore the error for the current purposes of my lab as long as the bindings are present:

silvia@vMX-ubuntu:~/vmx-15.1F4-3$ sudo ./vmx.sh --bind-check
Checking package ethtool..........................[OK]
Check Link vmx_link1(ge-0.0.0-vmx1, ge-0.0.1-vmx1)
[OK]
Check Link vmx_link2(ge-0.0.2-vmx1, ge-0.0.3-vmx1)
[OK]
Check Link vmx_link3(ge-0.0.4-vmx1, ge-0.0.5-vmx1)
[OK]
Check Link vmx_link4(ge-0.0.6-vmx1, ge-0.0.7-vmx1)
[OK]
Check Link vmx_link5(ge-0.0.8-vmx1, eth1).........[OK]
Check Link vmx_link6(ge-0.0.9-vmx1, eth2).........[OK]

Another interesting view of host ifconfig output.

silvia@vMX-ubuntu:~$ ifconfig | grep -A 1 encap
br-ext Link encap:Ethernet HWaddr 00:0c:29:9e:bb:5b 
      inet addr:192.168.83.10 Bcast:192.168.83.255 Mask:255.255.255.0
--
br-int-vmx1 Link encap:Ethernet HWaddr 52:54:00:fa:1a:e1 
      UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
--
eth0 Link encap:Ethernet HWaddr 00:0c:29:9e:bb:5b 
     inet6 addr: fe80::20c:29ff:fe9e:bb5b/64 Scope:Link
--
eth1 Link encap:Ethernet HWaddr 00:0c:29:9e:bb:65 
     inet addr:172.16.109.10 Bcast:172.16.109.255 Mask:255.255.255.0
--
eth2 Link encap:Ethernet HWaddr 00:0c:29:9e:bb:6f 
     inet addr:192.168.50.10 Bcast:192.168.50.255 Mask:255.255.255.0
--
ge-0.0.0-vmx1 Link encap:Ethernet HWaddr fe:06:0a:0e:ff:f0 
     inet6 addr: fe80::fc06:aff:fe0e:fff0/64 Scope:Link
--
ge-0.0.1-vmx1 Link encap:Ethernet HWaddr fe:06:0a:0e:ff:f1 
     inet6 addr: fe80::fc06:aff:fe0e:fff1/64 Scope:Link
--
ge-0.0.2-vmx1 Link encap:Ethernet HWaddr fe:06:0a:0e:ff:f2 
     inet6 addr: fe80::fc06:aff:fe0e:fff2/64 Scope:Link
--
ge-0.0.3-vmx1 Link encap:Ethernet HWaddr fe:06:0a:0e:ff:f3 
     inet6 addr: fe80::fc06:aff:fe0e:fff3/64 Scope:Link
--
ge-0.0.4-vmx1 Link encap:Ethernet HWaddr fe:06:0a:0e:ff:f4 
     inet6 addr: fe80::fc06:aff:fe0e:fff4/64 Scope:Link
--
ge-0.0.5-vmx1 Link encap:Ethernet HWaddr fe:06:0a:0e:ff:f5 
     inet6 addr: fe80::fc06:aff:fe0e:fff5/64 Scope:Link
--
ge-0.0.6-vmx1 Link encap:Ethernet HWaddr fe:06:0a:0e:ff:f6 
     inet6 addr: fe80::fc06:aff:fe0e:fff6/64 Scope:Link
--
ge-0.0.7-vmx1 Link encap:Ethernet HWaddr fe:06:0a:0e:ff:f7 
     inet6 addr: fe80::fc06:aff:fe0e:fff7/64 Scope:Link
--
ge-0.0.8-vmx1 Link encap:Ethernet HWaddr fe:06:0a:0e:ff:f8 
     inet6 addr: fe80::fc06:aff:fe0e:fff8/64 Scope:Link
--
ge-0.0.9-vmx1 Link encap:Ethernet HWaddr fe:06:0a:0e:ff:f9 
     inet6 addr: fe80::fc06:aff:fe0e:fff9/64 Scope:Link
--
lo Link encap:Local Loopback 
     inet addr:127.0.0.1 Mask:255.0.0.0
--
vcp_ext-vmx1 Link encap:Ethernet HWaddr fe:00:dd:c0:de:0e 
     inet6 addr: fe80::fc00:ddff:fec0:de0e/64 Scope:Link
--
vcp_int-vmx1 Link encap:Ethernet HWaddr fe:54:00:1f:5d:9c 
     inet6 addr: fe80::fc54:ff:fe1f:5d9c/64 Scope:Link
--
vfp_ext-vmx1 Link encap:Ethernet HWaddr fe:00:dd:c0:de:10 
     inet6 addr: fe80::fc00:ddff:fec0:de10/64 Scope:Link
--
vfp_int-vmx1 Link encap:Ethernet HWaddr fe:54:00:f1:04:f1 
     inet6 addr: fe80::fc54:ff:fef1:4f1/64 Scope:Link
--
virbr0 Link encap:Ethernet HWaddr 52:54:00:07:ea:ed 
     inet addr:192.168.122.1 Bcast:192.168.122.255 Mask:255.255.255.0
--
vmx_link1 Link encap:Ethernet HWaddr fe:06:0a:0e:ff:f0 
     inet6 addr: fe80::3cb1:36ff:fe47:58ee/64 Scope:Link
--
vmx_link2 Link encap:Ethernet HWaddr fe:06:0a:0e:ff:f2 
     inet6 addr: fe80::c432:7dff:fe91:f80e/64 Scope:Link
--
vmx_link3 Link encap:Ethernet HWaddr fe:06:0a:0e:ff:f4 
     inet6 addr: fe80::1cf7:7dff:fead:7ef5/64 Scope:Link
--
vmx_link4 Link encap:Ethernet HWaddr fe:06:0a:0e:ff:f6 
     inet6 addr: fe80::54f2:5fff:fe72:6601/64 Scope:Link
--
vmx_link5 Link encap:Ethernet HWaddr 00:0c:29:9e:bb:65 
     inet6 addr: fe80::cb1:6ff:fe88:a4e1/64 Scope:Link
--
vmx_link6 Link encap:Ethernet HWaddr 00:0c:29:9e:bb:6f 
     inet6 addr: fe80::2c26:b8ff:fed3:fd8d/64 Scope:Link

7. Connect to VFP using serial console

I will check my VFP serial console connectivity using user root and password root as configured default.

silvia@vMX-ubuntu:~/vmx-15.1F4-3$ sudo ./vmx.sh --console vfp vmx1 
--
Login Console Port For vfp-vmx1 - 8602
Press Ctrl-] to exit anytime
--
Trying ::1...
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.


Wind River Linux 6.0.0.12 vfp-vmx1 console

vfp-vmx1 login: root
Password: root

Here I can check logs:

root@vfp-vmx1:~# cd /var/log/ 
root@vfp-vmx1:/var/log# ls
boot riot.lcore.15.log riot.lcore.33.log riot.lcore.51.log
lastlog riot.lcore.16.log riot.lcore.34.log riot.lcore.52.log
messages riot.lcore.17.log riot.lcore.35.log riot.lcore.53.log
messages.0 riot.lcore.18.log riot.lcore.36.log riot.lcore.54.log
messages.1 riot.lcore.19.log riot.lcore.37.log riot.lcore.55.log
messages.2 riot.lcore.2.log riot.lcore.38.log riot.lcore.56.log
messages.3 riot.lcore.20.log riot.lcore.39.log riot.lcore.57.log
messages.4 riot.lcore.21.log riot.lcore.4.log riot.lcore.58.log
pfe.log riot.lcore.22.log riot.lcore.40.log riot.lcore.59.log
pfe.log.0 riot.lcore.23.log riot.lcore.41.log riot.lcore.6.log
pfe.log.1 riot.lcore.24.log riot.lcore.42.log riot.lcore.60.log
pfe.log.2 riot.lcore.25.log riot.lcore.43.log riot.lcore.61.log
pfe.log.3 riot.lcore.26.log riot.lcore.44.log riot.lcore.62.log
riot.lcore.0.log riot.lcore.27.log riot.lcore.45.log riot.lcore.63.log
riot.lcore.1.log riot.lcore.28.log riot.lcore.46.log riot.lcore.7.log
riot.lcore.10.log riot.lcore.29.log riot.lcore.47.log riot.lcore.8.log
riot.lcore.11.log riot.lcore.3.log riot.lcore.48.log riot.lcore.9.log
riot.lcore.12.log riot.lcore.30.log riot.lcore.49.log wtmp
riot.lcore.13.log riot.lcore.31.log riot.lcore.5.log
riot.lcore.14.log riot.lcore.32.log riot.lcore.50.log

!!! Some logs show up into this console. The can’t open ‘/var/jnx/card/local/type’ portion I remember from Junos vMX Router on VMWare Fusion article I’ve written 2 days ago. Interesting!

RPIO: Lost connection from localhost <-> localhost:3001
start_riot.sh: line 105: 3020 Killed /home/pfe/riot/build/app/riot -c 0x7 -n 2 -- --rx "(0,0,0,1),(1,0,1,1),(2,0,2,1),(3,0,3,1),(4,0,4,1),(5,0,5,1),(6,0,6,1),(7,0,7,1),(8,0,8,1),(9,0,9,1)," --tx "(0,1),(1,1),(2,1),(3,1),(4,1),(5,1),(6,1),(7,1),(8,1),(9,1)," --w "2" --rpio "local,3000,3001" --hostif "local,3002" --bsz "(32,32),(32,32),(32,32)"
mpc :
cat: can't open '/var/jnx/card/local/type': No such file or directory
tx_hello_tx: Failed to get card type defaulting to 0
cat: can't open '/var/jnx/card/local/slot': No such file or directory
tx_hello_tx: Failed to get card slot defaulting to 0
tnp_hello_tx: Board type 0
tnp_hello_tx: Board slot 0
SETTING UP DPDK ENVIRONMENT ON
Linux vfp-vmx1 3.10.38-ltsi-rt34-WR6.0.0.12_preempt-rt #1 SMP PREEMPT RT Wed Dec 2 16:34:33 PST 2015 x86_64 GNU/Linux
dpdk_setup.sh: line 21: make: command not found
insmod: can't insert '/home/pfe/riot/dpdk/x86_64-native-linuxapp-gcc/kmod/igb_uio.ko': File exists
Creating /mnt/huge and mounting as hugetlbfs
Detected Junos Device type VirtIO
bind interface 00:05.0 00:06.0 00:07.0 00:08.0 00:09.0 00:0a.0 00:0b.0 00:0c.0 00:0d.0 00:0e.0 from UIO
uio device registered with irq 2e
uio device registered with irq 2f
uio device registered with irq 30
uio device registered with irq 31
uio device registered with irq 32
uio device registered with irq 33
uio device registered with irq 34
uio device registered with irq 35
uio device registered with irq 36
uio device registered with irq 37
OK
kernel.core_pattern = /var/crash/core.%e.%t.%p.gz
Number of CPUs available : 3
Number of PORTs to service : 10
/home/pfe/riot/build/app/riot -c 0x7 -n 2 -- --rx "(0,0,0,1),(1,0,1,1),(2,0,2,1),(3,0,3,1),(4,0,4,1),(5,0,5,1),(6,0,6,1),(7,0,7,1),(8,0,8,1),(9,0,9,1)," --tx "(0,1),(1,1),(2,1),(3,1),(4,1),(5,1),(6,1),(7,1),(8,1),(9,1)," --w "2" --rpio "local,3000,3001" --hostif "local,3002" --bsz "(32,32),(32,32),(32,32)"
EAL: Setting up memory...
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7f4057600000 (size = 0x200000)
EAL: Ask a virtual area of 0x9400000 bytes
EAL: Virtual area found at 0x7f404e000000 (size = 0x9400000)
EAL: Ask a virtual area of 0x3b800000 bytes
EAL: Virtual area found at 0x7f4012600000 (size = 0x3b800000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7f4012200000 (size = 0x200000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7f4011e00000 (size = 0x200000)
EAL: Ask a virtual area of 0xe400000 bytes
EAL: Virtual area found at 0x7f4003800000 (size = 0xe400000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7f4003400000 (size = 0x200000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7f4003000000 (size = 0x200000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7f4002c00000 (size = 0x200000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7f4002800000 (size = 0x200000)
EAL: Ask a virtual area of 0x400000 bytes
EAL: Virtual area found at 0x7f4002200000 (size = 0x400000)
EAL: Ask a virtual area of 0x2400000 bytes
EAL: Virtual area found at 0x7f3fffc00000 (size = 0x2400000)
EAL: Ask a virtual area of 0x800000 bytes
EAL: Virtual area found at 0x7f3fff200000 (size = 0x800000)
EAL: Ask a virtual area of 0x1400000 bytes
EAL: Virtual area found at 0x7f3ffdc00000 (size = 0x1400000)
EAL: Ask a virtual area of 0x400000 bytes
EAL: Virtual area found at 0x7f3ffd600000 (size = 0x400000)
EAL: Ask a virtual area of 0x3400000 bytes
EAL: Virtual area found at 0x7f3ffa000000 (size = 0x3400000)
EAL: Ask a virtual area of 0x800000 bytes
EAL: Virtual area found at 0x7f3ff9600000 (size = 0x800000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7f3ff9200000 (size = 0x200000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7f3ff8e00000 (size = 0x200000)
EAL: Ask a virtual area of 0x400000 bytes
EAL: Virtual area found at 0x7f3ff8800000 (size = 0x400000)
EAL: Ask a virtual area of 0x600000 bytes
EAL: Virtual area found at 0x7f3ff8000000 (size = 0x600000)
EAL: Ask a virtual area of 0x400000 bytes
EAL: Virtual area found at 0x7f3ff7a00000 (size = 0x400000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7f3ff7600000 (size = 0x200000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7f3ff7200000 (size = 0x200000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7f3ff6e00000 (size = 0x200000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7f3ff6a00000 (size = 0x200000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7f3ff6600000 (size = 0x200000)
EAL: Ask a virtual area of 0x600000 bytes
EAL: Virtual area found at 0x7f3ff5e00000 (size = 0x600000)
EAL: Ask a virtual area of 0xc00000 bytes
EAL: Virtual area found at 0x7f3ff5000000 (size = 0xc00000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7f3ff4c00000 (size = 0x200000)
EAL: Ask a virtual area of 0x400000 bytes
EAL: Virtual area found at 0x7f3ff4600000 (size = 0x400000)
EAL: Ask a virtual area of 0x600000 bytes
EAL: Virtual area found at 0x7f3ff3e00000 (size = 0x600000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7f3ff3a00000 (size = 0x200000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7f3ff3600000 (size = 0x200000)
EAL: Requesting 768 pages of size 2MB from socket 0
EAL: TSC frequency is ~2903905 KHz
EAL: WARNING: cpu flags constant_tsc=yes nonstop_tsc=no -> using unreliable clock cycles !
EAL: 0000:00:03.0 not managed by UIO driver, skipping
EAL: 0000:00:04.0 not managed by UIO driver, skipping
CONFIG: --f argument not given : Flow caching will not be enabled
INIT: Creating the mbuf pool for socket 0 ...
INIT: Creating ring with priority 0 to connect I/O lcore 1 (socket 0) with worker lcore 2 ...
INIT: Creating ring with priority 1 to connect I/O lcore 1 (socket 0) with worker lcore 2 ...
INIT: Creating ring to connect worker lcore 2 with TX port 0 (through I/O lcore 1) (socket 0) ...
INIT: Creating ring to connect worker lcore 2 with TX port 1 (through I/O lcore 1) (socket 0) ...
INIT: Creating ring to connect worker lcore 2 with TX port 2 (through I/O lcore 1) (socket 0) ...
INIT: Creating ring to connect worker lcore 2 with TX port 3 (through I/O lcore 1) (socket 0) ...
INIT: Creating ring to connect worker lcore 2 with TX port 4 (through I/O lcore 1) (socket 0) ...
INIT: Creating ring to connect worker lcore 2 with TX port 5 (through I/O lcore 1) (socket 0) ...
INIT: Creating ring to connect worker lcore 2 with TX port 6 (through I/O lcore 1) (socket 0) ...
INIT: Creating ring to connect worker lcore 2 with TX port 7 (through I/O lcore 1) (socket 0) ...
INIT: Creating ring to connect worker lcore 2 with TX port 8 (through I/O lcore 1) (socket 0) ...
INIT: Creating ring to connect worker lcore 2 with TX port 9 (through I/O lcore 1) (socket 0) ...
INIT: No. of Lpolicer Buckets : 1
INIT: Setting worker Lpolicer: 0 start-idx 0
HOSTIF: Creating ring to connect hostif lcore 0 with worker lcore 2 ...
HOSTIF: Creating ring with priority 0 to connect worker lcore 0 (socket 0) with hostif worker lcore 2 ...
HOSTIF: Creating ring with priority 1 to connect worker lcore 0 (socket 0) with hostif worker lcore 2 ...
CONFIG: Runtime option summary:
CONFIG: rx capture port mask : 0x0
CONFIG: rx capture frequency : 1
CONFIG: rx capture length : 128
EAL: 0000:00:03.0 not managed by UIO driver, skipping
EAL: 0000:00:04.0 not managed by UIO driver, skipping
INIT: Initializing NIC port 0 ...
INIT: Initializing NIC port 0 RX queue 0 ...
INIT: Initializing NIC port 0 TX queue 0 ...
INIT: Initializing NIC port 1 ...
INIT: Initializing NIC port 1 RX queue 0 ...
INIT: Initializing NIC port 1 TX queue 0 ...
INIT: Initializing NIC port 2 ...
INIT: Initializing NIC port 2 RX queue 0 ...
INIT: Initializing NIC port 2 TX queue 0 ...
INIT: Initializing NIC port 3 ...
INIT: Initializing NIC port 3 RX queue 0 ...
INIT: Initializing NIC port 3 TX queue 0 ...
INIT: Initializing NIC port 4 ...
INIT: Initializing NIC port 4 RX queue 0 ...
INIT: Initializing NIC port 4 TX queue 0 ...
INIT: Initializing NIC port 5 ...
INIT: Initializing NIC port 5 RX queue 0 ...
INIT: Initializing NIC port 5 TX queue 0 ...
INIT: Initializing NIC port 6 ...
INIT: Initializing NIC port 6 RX queue 0 ...
INIT: Initializing NIC port 6 TX queue 0 ...
INIT: Initializing NIC port 7 ...
INIT: Initializing NIC port 7 RX queue 0 ...
INIT: Initializing NIC port 7 TX queue 0 ...
INIT: Initializing NIC port 8 ...
INIT: Initializing NIC port 8 RX queue 0 ...
INIT: Initializing NIC port 8 TX queue 0 ...
INIT: Initializing NIC port 9 ...
INIT: Initializing NIC port 9 RX queue 0 ...
INIT: Initializing NIC port 9 TX queue 0 ...
RPIO: Command socket listening on: localhost:3000
RPIO: Event socket listening on: localhost:3001
LU: Initializing LU
INIT: Initialization completed.
CONFIG: NIC RX ports: CONFIG: 0 (CONFIG: 0 CONFIG: ) CONFIG: 1 (CONFIG: 0 CONFIG: ) CONFIG: 2 (CONFIG: 0 CONFIG: ) CONFIG: 3 (CONFIG: 0 CONFIG: ) CONFIG: 4 (CONFIG: 0 CONFIG: ) CONFIG: 5 (CONFIG: 0 CONFIG: ) CONFIG: 6 (CONFIG: 0 CONFIG: ) CONFIG: 7 (CONFIG: 0 CONFIG: ) CONFIG: 8 (CONFIG: 0 CONFIG: ) CONFIG: 9 (CONFIG: 0 CONFIG: ) CONFIG: ;
CONFIG: I/O lcore 1 (socket 0): CONFIG: RX ports CONFIG: (0, 0, 0) CONFIG: (1, 0, 1) CONFIG: (2, 0, 2) CONFIG: (3, 0, 3) CONFIG: (4, 0, 4) CONFIG: (5, 0, 5) CONFIG: (6, 0, 6) CONFIG: (7, 0, 7) CONFIG: (8, 0, 8) CONFIG: (9, 0, 9) CONFIG: ; CONFIG: Output rings CONFIG:
Priority : HiCONFIG: 0x7f40577ea000 CONFIG:
Priority : NorCONFIG: 0x7f40577ec080 CONFIG: ;
CONFIG: Worker lcore 2 (socket 0) ID 0: CONFIG: Input rings CONFIG:
Priority : HiCONFIG: 0x1d9a540 CONFIG: 0x1d9a608 CONFIG:
Priority : NorCONFIG: 0x1d9a540 CONFIG: ;
CONFIG:
CONFIG:
CONFIG: NIC TX ports: CONFIG: 0 CONFIG: 1 CONFIG: 2 CONFIG: 3 CONFIG: 4 CONFIG: 5 CONFIG: 6 CONFIG: 7 CONFIG: 8 CONFIG: 9 CONFIG: ;
CONFIG: I/O lcore 1 (socket 0): CONFIG: Input rings per TX port CONFIG: 0 (CONFIG: 0x7f40577ee100 CONFIG: ) CONFIG: 1 (CONFIG: 0x7f40577f0180 CONFIG: ) CONFIG: 2 (CONFIG: 0x7f40577f2200 CONFIG: ) CONFIG: 3 (CONFIG: 0x7f40577f4280 CONFIG: ) CONFIG: 4 (CONFIG: 0x7f40577f6300 CONFIG: ) CONFIG: 5 (CONFIG: 0x7f40577f8380 CONFIG: ) CONFIG: 6 (CONFIG: 0x7f40577fa400 CONFIG: ) CONFIG: 7 (CONFIG: 0x7f40577fc480 CONFIG: ) CONFIG: 8 (CONFIG: 0x7f3ff5b8c740 CONFIG: ) CONFIG: 9 (CONFIG: 0x7f3ff5b8e7c0 CONFIG: ) CONFIG: ;
CONFIG: Worker lcore 2 (socket 0) ID 0:
CONFIG: Output rings per TX port CONFIG: 0 (0x7f40577ee100) CONFIG: 1 (0x7f40577f0180) CONFIG: 2 (0x7f40577f2200) CONFIG: 3 (0x7f40577f4280) CONFIG: 4 (0x7f40577f6300) CONFIG: 5 (0x7f40577f8380) CONFIG: 6 (0x7f40577fa400) CONFIG: 7 (0x7f40577fc480) CONFIG: 8 (0x7f3ff5b8c740) CONFIG: 9 (0x7f3ff5b8e7c0) CONFIG: ;
CONFIG: Ring sizes: NIC RX = 1024; Worker in = 1024; Worker out = 1024; NIC TX = 1024;
CONFIG: Burst sizes: I/O RX (rd = 32, wr = 32); Worker (rd = 32, wr = 32); I/O TX (rd = 32, wr = 32)
RUNTIME: Logical core 1 (I/O) main loop.

RUNTIME: Logical core 2 (worker 0) main loop.

RPIO: Accepted connection from localhost <-> localhost:3000
RPIO: Accepted connection from localhost <-> localhost:3001
RIOT: Received bandwidth config: b/w : 6250000

RIOT: Initializing policer for bank 0, bucket : 0 rate: 6250000

METER: Low level srTCM config:
CIR period = 464, CIR bytes per period = 1
RIOT: New policer index: 0

HOSTIF: Accepted connection

RUNTIME: Detected port 0 status changed to UP

RUNTIME: Detected port 1 status changed to UP

RUNTIME: Detected port 2 status changed to UP

RUNTIME: Detected port 3 status changed to UP

RUNTIME: Detected port 4 status changed to UP

RUNTIME: Detected port 5 status changed to UP

RUNTIME: Detected port 6 status changed to UP

RUNTIME: Detected port 7 status changed to UP

RUNTIME: Detected port 8 status changed to UP

RUNTIME: Detected port 9 status changed to UP

8. Connect to VCP using serial console

The default configuration is user root with no password. I have already configured the vMX in my old laboratories.

silvia@vMX-ubuntu:~/vmx-15.1F4-3$ sudo ./vmx.sh --console vcp vmx1
[sudo] password for silvia:

vMX (ttyd0)

login: root
password: ********

--- JUNOS 15.1F4.15 built 2015-12-23 20:22:39 UTC

root@vMX% cli
root@vMX>

9. Configure SSH connectivity

I configure fxp0 interface with the OoB Management IP address:

[edit]
root@vMX# show interfaces fxp0 
unit 0 {
      family inet {
            address 192.168.83.11/24;
      }
}

[edit]
root@vMX# run show route

inet.0: 2 destinations, 2 routes (2 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

192.168.83.0/24 *[Direct/0] 00:15:24
 > via fxp0.0
192.168.83.11/32 *[Local/0] 00:15:24
 Local via fxp0.0

Let’s test some ping:

[edit]
root@vMX# run ping 192.168.83.10  <<-- to Ubuntu Host
PING 192.168.83.10 (192.168.83.10): 56 data bytes
64 bytes from 192.168.83.10: icmp_seq=0 ttl=64 time=0.335 ms
^C
--- 192.168.83.10 ping statistics ---
4 packets transmitted, 4 packets received, 0% packet loss
round-trip min/avg/max/stddev = 0.270/0.352/0.430/0.058 ms

[edit]
root@vMX# run ping 192.168.83.1  <<-- to MacBook 
PING 192.168.83.1 (192.168.83.1): 56 data bytes
64 bytes from 192.168.83.1: icmp_seq=0 ttl=64 time=1.249 ms
^C
--- 192.168.83.1 ping statistics ---
2 packets transmitted, 2 packets received, 0% packet loss
round-trip min/avg/max/stddev = 0.591/0.920/1.249/0.329 ms

[edit]
root@vMX# run ping 192.168.83.12   <<-- to VFP
PING 192.168.83.12 (192.168.83.12): 56 data bytes
64 bytes from 192.168.83.12: icmp_seq=0 ttl=64 time=2.595 ms
^C
--- 192.168.83.12 ping statistics ---
2 packets transmitted, 2 packets received, 0% packet loss
round-trip min/avg/max/stddev = 0.613/1.604/2.595/0.991 ms

Let’s check if I can connect to vRE from Ubuntu Host:

silvia@vMX-ubuntu:~$ ssh root@192.168.83.11 
The authenticity of host '192.168.83.11 (192.168.83.11)' can't be established.
ECDSA key fingerprint is 34:0a:47:46:92:0f:f9:ba:8d:e3:99:9a:bd:3c:82:71.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '192.168.83.11' (ECDSA) to the list of known hosts.
Password:
Last login: Fri Nov 17 11:21:06 2017
--- JUNOS 15.1F4.15 built 2015-12-23 20:22:39 UTC
root@vMX% cli
root@vMX>

BINGO!! Now I can connect also from my MacBook using SecureCRT.

10. Create and save a vMX base configuration file vmx_base.conf

Next, I create and I save into my SkyLifter project folder a base configuration file using file path: SkyLifter/SkylifterSource/vmx_base.conf.

11. Comand Line Summary

cd vmx-15.1F4-3/

sudo ./vmx.sh -lv --install

brctl show

sudo ./vmx.sh --bind-check

sudo ./vmx.sh --bind-dev

sudo ./vmx.sh --bind-check

sudo ./vmx.sh --console vfp vmx1 (root, root)

sudo ./vmx.sh --console vcp vmx1 (root, SilviaMurgescu)

Junos vMX Router on VMWare Fusion

I have successfully installed vMX and used it in my labs following the procedure describer into Day One: vMX Up and Running book. But I am curious to see if the installing procedure used for IOS XRv can be used also for the new Nested vMX router from Juniper Networks.

I am still having some trouble with some errors. Maybe it is a resource limitation. I will update the article if I find a solution to make it up and running.
Continue reading