Background

In 802.1Q networks VLANs provide isolation between different sets of end stations. An end station on one VLAN is unable to transmit a frame to an end station on another VLAN. In a VLAN attack this rule is broken by exploiting vulnerabilities in the bridge software or hardware. Over the years various such attacks have been discovered and resolved, and properly configured VLANs are now considered secure.

Today, the most important and widely used control when securing VLANs is the ability to constrain their topology. This is a well defined feature in the 802.1Q standard. A VLAN topology is constrained by configuring VLAN memberships on ports. A bridge only accepts a frame on a port when the frame’s VLAN tag matches the port’s VLAN memberships. Similarly, a bridge only transmits a frame on a port — such as in the case of a broadcast flood — when there is also a match.

Let’s consider the following simple example.

   +----+         +----+         +----+
   | B1 |--dot1Q--| B2 |--dot1Q--| B3 |
   +----+         +----+         +----+

There are three 802.1Q bridges. B1 is in an insecure location, such as a meeting room or wiring closet, where an attacker might gain physical access and compromise it. B2 and B3 are in a secure location, such as a server room, and have ports configured on VLAN 2 with servers attached. The administrator wants B1 to have access to VLAN 1, and to prevent a rogue B1 from being able to access the servers on VLAN 2. So on B2, he configures the port connected to B1 to only be a member of VLAN 1. Now, if an attacker compromises B1, he can configure VLAN 2 on the compromised bridge, but this access will not extend to the target servers. To do this the attacker would need to gain access to and compromise B2, which is considered to be in a secure location.

Enter TRILL

So how might TRILL make a network vulnerable? Essentially the problem is that TRILL bypasses the existing VLAN topology (or policy) of a network. When an Ethernet frame arrives at an RBridge it’s encapsulated with a TRILL header and an outer MAC header. But, importantly, the VLAN tag isn’t copied to the outer MAC header, and instead TRILL forwards the frame over a single common VLAN called the designated VLAN.

Going back to the example, if B1 and B3 are replaced with RBridges, one might naturally expect a TRILL VLAN to behave like an 802.1Q VLAN, so that VLAN 2 will still be constrained by the configuration on B2, but this is not the case. An attacker that compromised B1 could configure full access to any end station on VLAN 2 attached to B2 and B3. In fact, the attacker could configure full access to any VLAN which any other RBridge in the network can access. This is because B1 will encapsulate VLAN 2 frames from the attacker, forward them to B3, where they are decapsulated and forwarded appropriately. B2 is unable to recognise the encapsulated VLAN 2 frame so lets it pass.

Other scenarios in which TRILL may enable VLAN attacks may also exist. For example, consider the case where end stations exist on the designated VLAN. Can an end station then emulate or spoof an RBridge, and therefore gain access to the entire network? In some cases, yes.

Conclusion

TRILL has adopted the term VLAN, but what it calls a VLAN behaves differently from the 802.1Q definition of a VLAN. TRILL views a VLAN as a community, more like a BGP/MPLS IP VPN, and believes that all end stations in a VLAN should be able to reach each other. TRILL also takes the view that all RBridges in a network are at the same trust level. Again, this is similar to MPLS, and we see that no one ever puts an MPLS router in an insecure location for that reason. TRILL goes so far as to impose this view on existing 802.1Q networks when it is deployed incrementally.

So what could be done? Well, there are only two ways to properly fix this.

The first is for TRILL to copy the inner VLAN tag to the outer VLAN tag, so that a TRILL encapsulated frame could still only travel on the same links as before RBridges were deployed. As an advantage, the VLAN forwarding behaviour of RBridges and bridges remain similar.

The second is for TRILL to form regions of directly coupled RBridges, where TRILL is only done on RBridge-RBridge links. This would require making these TRILL regions appear to STP as though they are bridges through some clever processing of BPDUs, or making the TRILL regions transparently tunnel BPDUs. This would have the added benefit of being similar to existing systems, such as Multiple Spanning Tree, 802.1ad Provider Bridges, and VPLS.

It is likely that implementations of RBridges will allow administrative filtering of the inner VLAN, but this is not included in the specification and so may vary. (At a minimum I feel this filtering should be covered in the specification.) This will allow administrators, provided they understand TRILL properly and deploy RBridges in the right sequence, to mimic their existing policy.

It strikes me as strange that a protocol aimed at resolving many of the problems with bridges has chosen to create new, preventable problems. It has chosen to adopt existing terms but behave differently, and has opted for the best incremental optimisation outcome over respecting the existing security policy. I think it’s likely that we will see RBridges sold as drop-in replacements for bridges, and I remain concerned that in the early days, because of this, there will misunderstandings and problems.

The proposal seems to believe that wholesale unbundled dark fibre is the best method to serve all customers, which is absolutely incorrect. The negative impacts identified in this submission are caused by this, and forcing LFCs to engineer the networks in such a way.

Unbundled dark fibre is only suitable for a small percentage of customers and is completely unsuitable for the remaining customers that make up the vast majority. It is important that this distinction is understood, and that the proposal targets the distinct needs of these two categories of customer.

The full 21 page submission is available in PDF format here.

An abridged history of the RBridge header
Written by Kris Price, 10 March 2009.

Note: Names and references have been excluded to prevent anyone being unwillingly associated with my interpretation of events. Naturally I have no visibility of meetings or private discussions. The details can be found by reading the TRILL mailing list archive of the period from October to December, 2006.

TRILL initially looked to use MPLS based encapsulation for RBridges, as described in draft-bryant-perlman-trill-pwe-encap-00. The approach created a global MPLS labelspace from which RBridges selected their nicknames. The MPLS header would only include egress RBridge, CoS, and TTL information. An RBridge would make additional forwarding decisions, for example to constrain delivery of a VLAN tagged frame to only those links on which that VLAN is active, by using the information in the encapsulated MAC header. This was a deviation from the general architecture of MPLS that such forwarding decisions are made at the edge by selecting the correct label switch path. The responsibility falling to the control-plane to set up the necessary label switched paths accordingly. Hence the amount of hardware re-use at the time may have been limited depending on whether this filtering became a strict requirement or an option that could be phased in as new hardware was developed. Because this type of filtering has unrelated benefits it may well find its way into many platforms anyway, and today some platforms may already have some of this capability for that reason.

A potential requirement for the inclusion of ingress RBridge information within the header was raised on the mailing list in October 2006. It was based on the need to possibly support 802.1au Congestion Notification, in which notifications must be returned to the end-station that sent the triggering frame. To do this, it was argued that an RBridge trying to send such a notification would need to know the ingress RBridge. It was suggested that the ingress RBridge could be identified by looking at the source MAC address in the encapsulated MAC header. One group felt it was inappropriate for a core RBridge to maintain all of the MAC reachability information, as would be required in order to do this lookup. Another group felt this was reasonable, as that is what core IEEE bridges to today. In the case where all of the MAC reachability information was not maintained, the approach of sending the notification over the multi-destination delivery tree (since it is then effectively an unknown destination frame) was rejected as having negative impacts.

Other fields were also proposed for inclusion in the header at this time, as described in draft-gai-perlman-trill-encap-00, but these were eventually ruled out, or incorporated in other ways. The resulting approach that was settled on was to include ingress and egress RBridge information. This was seen as more natural and flexible to future requirements. The option of continuing to use MPLS based encapsulation was raised, using a label stack, with the top label as per the original proposal, and the bottom label identifying the ingress RBridge. This appears to have received very little attention. I can only speculate that it was rejected out of a belief that MPLS based encapsulation brought little benefit, in which case a simplified 6-byte header versus an 8-byte header was more elegant.

The use of the encapsulation defined for 802.1ah Provider Backbone Bridges was raised at the IETF-67 meeting in November, 2006. At the time 802.1ah was on its way to becoming a standard, and the question was whether there were any synergies between it and RBridges that could be capitalised on. Subsequent discussions on the mailing list ultimately rejected the idea for a few reasons. There was a feeling that 802.1ah was targeted at the provider market and so unsuitable for the enterprise and data centre market, although it is unclear why this made it unsuitable. There was a larger concern that 802.1ah would require a partial move away from zero configuration, although this was never clearly detailed. The clearest arguments against the use of 802.1ah were the lack of a TTL in the header, and the lack of a next-hop address. It was suggested that a TTL could be accommodated in the header before the standard was finalised, but the lack of a next-hop address was unresolved. RBridges uses the outer MAC encapsulation to define the next-hop on each link, this allows load balancing to multiple next-hop RBridges across a shared link. It is unclear whether the benefits of this versus the benefits of a shared encapsulation were weighed.

When 802.1ah based encapsulation was proposed it was noted that by using a non-IEEE frame format TRILL would be assuming responsibility for implementing all of the features developed by IEEE.

Subsequent to this the header acquired the options fields to permit extensions, and it has resulted in the TRILL header seen today.

The end.

After watching this I became very interested in why MPLS was not used. MPLS appears to naturally lend itself to this problem, and shares similarities with the approach ultimately adopted by TRILL. Why use MPLS? Mainly for the re-use of existing hardware. Adding a new control-plane is much simpler as it is predominantly a matter of software development. Re-using this hardware would reduce development costs and the time to market.

At the time I attempted to find out why this was, but no clear answers were forthcoming. I became distracted, but the subject never fully left my mind. After pondering it at some length I thought I would summarise the basics here, in case it interests other people too.

Overview

1. Routers run a link-state protocol, building a topological database from which they identify the shortest paths to other routers. Routers set up label switched paths (LSPs) to the other routers.
2. Routers use either control-plane or data-plane learning of reachability information. In control-plane learning, routers learn the MAC addresses of active end stations on attached links, as a normal IEEE bridge would, and distribute this over the link-state protocol so that sending routers can identify the appropriate LSPs on which to forward frames. In data-plane learning, each router learns the appropriate LSPs by examining packets during decapsulation.
3. When a frame from an end station arrives at a router, the router looks up the destination MAC address, identifying the egress router and appropriate label switch path. The router encapsulates the frame in an MPLS header, and the encapsulated frame is then label switched across the network to the egress node. At the egress node the label is popped and the frame forwarded out the appropriate interface.

Forwarding

Single-destination frames would be forwarded on multipoint-to-point (MP2P) LSPs. Point-to-point (P2P) LSPs do not scale as well. For single-destination forwarding, scaling is essentially O(n) using MP2P LSPs compared to O(n^2) using P2P LSPs.

Multi-destination forwarding is trickier, with many variations on approaches that could be adopted. It depends on clearly identifying how VLAN tagged multi-destination frames and IP multicast is to be handled. Then balancing the trade offs between bending the rules in the MPLS architecture, any requirements for data-plane learning, and the intricacy of the control-plane. Trees would need to be set up, either shared trees using multipoint-to-multipoint (MP2MP) LSPs, or source-based trees using point-to-multipoint (P2MP) LSPs.

Constraining delivery of multi-destination frames

Constraining the delivery of multi-destination frames, to prevent unnecessary resource consumption in the case of VLANs and IP multicast, would fall to setting up additional trees or using hardware filtering (a feature many modern platforms either have, could adopt, or will adopt for other purposes anyway).

Some form of multi-topology routing could be used to discover the topology of each VLAN, and when transmitting across each link the outer MAC header would contain the VLAN tag of the topology in question, on receipt the router can use this additional information to classify the packet. This can be seen to increase the MPLS label space (“per-subinterface”) and constrain the forwarding of frames in a VLAN to links which are in those VLANs. Directly coupled router to router links would simply be full 802.1Q trunks.

Another option is not to be concerned about constraining frames to only those links which allow those VLANs, and each router advertises VLAN membership, with constrained distribution trees established for each VLAN. The outer VLAN tags used on each link would not be relevant. This is the approach taken by TRILL, and it is distinctly different from 802.1Q behaviour (but that is for another post).

IP multicast can also be optimised with constrained trees, and the multicast group membership information distributed by the link-state protocol.

Control-plane

I did discover that MPLS encapsulation was proposed for RBridges in draft-bryant-perlman-trill-pwe-encap-00, which violates the MPLS architecture, because label assignments are made upstream, but that upstream allocation does certainly make things easier for the control-plane in a system like this, and it is an easy violation to live with. A new ethertype is used in this case to distinguish a new global labelspace.

The draft does discuss how the paths would be established and it is possible to conceive ways of establishing additional constrained trees for VLANs and IP multicast optimisation too. However, after cursory examination of methods for this, it would seem to require routers to do SPF computations for all routers in the network. A router would need to know its position in the tree to decide if it needed to install relevant forwarding state.

A signalling protocol in the control-plane (perhaps separate, or perhaps carried on the link-state protocol) could remove the need to do these SPF computations, and remove the upstream assignment of labels, making it compliant to the MPLS architecture. But this would be a trade off in that the resulting protocol would become more intricate.

Learning

It appears from the experience of TRILL that data-plane learning is a desirable feature. To achieve this, there are a few options that could be explored.

If the use of a global labelspace was undesirable, learning in the case of single-destination forwarding would be solved by placing an extra label in the stack when encapsulating a frame in MPLS. This can be used by the egress router to identify the ingress router. In essence these are P2P LSPs tunnelled over the MP2P LSPs. Learning in the case of multi-destination forwarding require that P2MP LSPs be used. Because a P2MP LSP has a single source, the egress router knows where the arriving packet was encapsulate.

The simpler option is to use the global labelspace, as discussed in draft-bryant-perlman-trill-pwe-encap-00. This would permit an ingress router to place an extra label in the stack identifying itself. Because the labelspace is global, and a protocol has permitted all routers to select their own unique label, they can safely use this to identify themselves for both single- and multi-destination packets using a label stack.

The final option is to look at using some kind of MAC-in-MAC encapsulation before the MPLS encapsulation, with the outer MAC header identifying the source and destination addresses of the routers in question (where only the source address is actually relevant). The egress router, after popping the MPLS label, would learn the ingress router via this outer MAC header. Again this works for both single- and multi-destination frames, but it is not very attractive. It would require more customised hardware. It is also stepping away from the use of MPLS. In this case, it would probably be more suitable to straight out use 802.1ah and that is already being explored by the IEEE’s Shortest Path Bridging effort. Many extensible MPLS platforms today have the ability to learn from labels. Those that do not could acquire this ability over the development time of the protocol.

That is the sum of it. The rest gets into the details of weighing these approaches against each other and designing the protocols.