EVPN Ansible playbook for Cumulus Linux 3.2.1

I wrote a quick Ansible playbook for Cumulus EVPN. Cumulus EVPN is now GA (Generally Available) but the packages are still in the EA (early access repo) so it can be confusing if you are not used to Debian packaging system. This is nice that it won’t try to upgrade/reboot unless you have the wrong version. Feel free to read the documentation on Cumulus Networks website.

 

- name: check current quagga version for EVPN
  command: "dpkg -l quagga"
  register: quaggaversion
  when: ansible_lsb.major_release == "3"

- name: debug quaggaversion
  debug:
     var: quaggaversion.stdout
  when: ansible_lsb.major_release == "3"

- name: uncomment early access repo
  lineinfile: >
    dest=/etc/apt/sources.list
    regexp="#deb     http://repo3.cumulusnetworks.com/repo CumulusLinux-3-early-access cumulus"
    line="deb     http://repo3.cumulusnetworks.com/repo CumulusLinux-3-early-access cumulus"
    state=present
  when: ansible_lsb.major_release == "3"


- name: uncomment early access repo sources
  lineinfile: >
    dest=/etc/apt/sources.list
    regexp="#deb-src http://repo3.cumulusnetworks.com/repo CumulusLinux-3-early-access cumulus"
    line="deb-src http://repo3.cumulusnetworks.com/repo CumulusLinux-3-early-access cumulus"
    state=present
  when: ansible_lsb.major_release == "3"

- name: install eau8 of quagga
  apt: name="cumulus-evpn" update_cache=yes
  when: ansible_lsb.major_release == "3" and "eau8" not in quaggaversion.stdout

- name: upgrade switch (part of EVPN install instructins)
  shell: "apt-get upgrade -y --force-yes -o Dpkg::Options::='--force-confnew'"
  become: true
  become_method: sudo
  when: 'ansible_lsb.major_release == "3" and "eau8" not in quaggaversion.stdout'

- name: Reboot for apt-get upgrade
  shell: sleep 2 && shutdown -r now "Ansible updates triggered"
  async: 1
  poll: 0
  become: true
  ignore_errors: true
  when: 'ansible_lsb.major_release == "3" and "Cumulus" in ansible_lsb.id and "eau8" not in quaggaversion.stdout'

- name: Wait for everything to come back up
  local_action: wait_for port=22 host="{{ inventory_hostname }}" search_regex=OpenSSH delay=10
  when: 'ansible_lsb.major_release == "3" and "eau8" not in quaggaversion.stdout'

Fully Qualified Domain Name Capability for BGP

I really like Daniel Walton‘s draft-walton-bgp-hostname-capability-00.  Cumulus Networks implemented this as part of Quagga/FRR and it has become to default on Cumulus Linux 3.0 and later.  BGP is one of those protocols that is really powerful but in the past is really a pain in the ass to troubleshoot if you don’t know how things are cabled.  Without a network map or diagram it will take awhile to reverse engineer and troubleshoot a customer’s network. 

This capability allows the configured BGP neighbor hostname to be shared via BGP (not lldp or cdp) so it can give you the output with the BGP commands themselves without having to correlate a neighbor to IP address, and IP address to physical switchport.  What does that look like?

cumulus@leaf01:mgmt-vrf:~$ net show bgp sum

show bgp ipv4 unicast summary
=============================
BGP router identifier 10.0.0.11, local AS number 65011 vrf-id 0
BGP table version 107
RIB entries 25, using 3400 bytes of memory
Peers 2, using 42 KiB of memory
Peer groups 1, using 72 bytes of memory

Neighbor        V         AS MsgRcvd MsgSent   TblVer  InQ OutQ  Up/Down State/PfxRcd
spine01(swp51)  4      65020  286176  286211        0    0    0 01w2d22h           11
spine02(swp52)  4      65020  224758  224766        0    0    0 4d18h12m           11

Wow… that is beautiful. Now I know my two uplinks (switchport 51 and switchport 52) are hooked up to spine01 and spine02. the output without this default (“no bgp default show-hostname”) looks like this:

cumulus@leaf01:mgmt-vrf:~$ net show bgp sum

show bgp ipv4 unicast summary
=============================
BGP router identifier 10.0.0.11, local AS number 65011 vrf-id 0
BGP table version 107
RIB entries 25, using 3400 bytes of memory
Peers 2, using 42 KiB of memory
Peer groups 1, using 72 bytes of memory

Neighbor        V         AS MsgRcvd MsgSent   TblVer  InQ OutQ  Up/Down State/PfxRcd
swp51           4      65020  286341  286376        0    0    0 01w2d22h           11
swp52           4      65020  224923  224931        0    0    0 4d18h21m           11

I use this for every customer network I work on (which is easy now that it is the default). I hope more and more vendors implement this capability because it is very slick.

Let me know what you think in the comments, I will talking about new features in BGP that we have been using for awhile but might not be that common outside of Cumulus Networks.

Turning on all Cumulus Linux interfaces

Cumulus Linux has a cool ‘feature’ it inherited from Linux.  On most network switches  a port is either Layer2 or Layer3.  When a port is layer2 it has to be part of a VLAN.  Linux does understand and work with VLANs but it can have a port that is running at layer2 but not part of a VLAN.  We can configure a port under /etc/network/interfaces like this:

auto swp1
iface swp1

If we ifup this port its not part of a VLAN, and its not Layer3.  

Why would we want to do this?  Honestly the most common reason is that we can check physical connections by using lldp before this port is part of a broadcast domain (VLAN) that could cause loops or unexpected behavior.

What if I cabled a switch and don’t know what ports are connected?  You could create a stanza for each swp like above, use something like mako or do a simple bash loop on the command line like this:

cumulus@leaf01:~$ for swp in {1..54}; do sudo ip link set swp$swp; done

Now all 54 ports are admin up at layer2 so we can check connections, but its not routing or switching.  Now you can use a “net show int” or use the linux command “ip link show”

cumulus@leaf01:~$ ip link show | grep LOWER_UP
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master mgmt state UP mode DEFAULT group default qlen 1000
11: swp9: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9216 qdisc pfifo_fast state UP mode DEFAULT group default qlen 1000
51: swp49: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 9216 qdisc pfifo_fast master peerlink state UP mode DEFAULT group default qlen 1000
52: swp50: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 9216 qdisc pfifo_fast master peerlink state UP mode DEFAULT group default qlen 1000
53: swp51: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9216 qdisc pfifo_fast state UP mode DEFAULT group default qlen 1000
54: swp52: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9216 qdisc pfifo_fast state UP mode DEFAULT group default qlen 1000

You can even just use lldp here

cumulus@leaf01:~$ net show lldp

Local Port    Speed    Mode                 Remote Port    Remote  Host     Summary
------------  -------  -------------  ----  -------------  ---------------  --------------------------
eth0          1G       Mgmt           ====  swp23          oob-mgmt-switch  IP: 10.50.100.100/24(DHCP)
swp9          10G      NotConfigured  ====  swp51          leaf07
swp49         40G      NotConfigured  ====  swp49          exit02           
swp50         40G      NotConfigured  ====  swp50          exit02           
swp51         40G      NotConfigured  ====  swp30          spine01
swp52         40G      NotConfigured  ====  swp30          spine02

Yay now we can go configure them now that we know how they are cabled up. Its great to be lazy 🙂

Update 3/30/2017

The famous Daniel Walton (yeah the Daniel that has his name on this, this and my personal favorite this.) let me know that Cumulus Linux’s new NCLU (Network Command Line Utility) also has a method of doing this quickly. You can check out his tweet here. NCLU uses the net command and has the ability to do a range of ports really quickly. One caveat vs using ip link set… this will make persistent config (meaning the ports are actually configured under /etc/network/interfaces). If you don’t want the config to remain you can do a “net del” then a “net commit”

net add int swp1-54

Quagga on Ubuntu – Triangle Network Engineers

On March 3rd, 2015 I presented to a small group (maybe 30 all together?) of Network Enthusiasts at Cumulus Networks East-Coast HQ (3701 NW Cary Parkway, Cary, NC) for the Triangle Network Engineers Meetup group.  If you are interested in future meetups check out one of these Meetup groups that focus on networking and automation:

 

Attached is my presentation (attached pdf) quagga-triangle-network-engineers

Was having one of those days….

You ever have one of those days where your laptop won’t boot your last backup is too old for comfort and then you boot your desktop backup and it’s toast as well?

20140609-222621-80781578.jpg

VRRPv3 with IPv6 Support for JUNOS & Cisco IOS

Ran into a problem the other day where I was trying to configure VRRPv3 (which supports IPv6) between a Juniper MX480 and a Cisco 6509-E w/ SUP2T. The only config guide I could find for the Cisco device was this. While fairly detailed on turning on VRRP, (and remember you need this command)

fhrp version vrrp v3

it did not go into any detail for the actual IPv6 implementation! I already pinged the appropriate people but I thought I would share my config below in the meantime. The main thing to note is that when configuring IPv6 w/ VRRPv3 you have to manually add the link locals (as of the 15.1.1 ~Nov 18th, 2012).

So I won’t put a diagram here but basically just imagine a port-channel between the Cisco & Juniper that only allows vlan 2100. Here is the Cisco config->

interface Vlan2100
 ip address 201.13.110.2 255.255.254.0
 ipv6 address 2201:13:110::2/64
 ipv6 enable
 vrrp 1 address-family ipv4
  priority 120
  address 201.13.110.1 primary
 exit-vrrp
 vrrp 1 address-family ipv6
  priority 120
  address FE80::13:110:1 primary
  address 2201:13:110::1/64
 exit-vrrp
end

So the Juniper will automatically create a link-local address but you can override it (which you have to do if you are doing it on the Cisco). Here is my Juniper config->

irb {
        unit 2100 {
            description "Voice Vlan";
            family inet {
                address 201.13.110.3/23 {
                    vrrp-group 1 {
                        virtual-address 201.13.110.1;
                        accept-data;
                    }
                }
            }
            family inet6 {
                address 2201:13:110::3/64 {
                    vrrp-inet6-group 1 {
                        virtual-inet6-address 2201:13:110::1;
                        virtual-link-local-address fe80::13:110:1;
                        accept-data;
                    }
                }

This config worked perfectly, and although it looks simple now it was pretty annoying to figure out that the routers were not smart enough to sync on their link-local if they have the same virtual global or vice versa. Hopefully this will help someone out there! If you have any questions, advice, etc make comments below. Thanks.