Hairpinning traffic through ASA with State Bypass

Several years ago I wrote an article about the Woes of Using an ASA as a Default Gateway. I have received a lot of feedback about this post and recently had a request for an update around ASA > 8.3. When building this scenario out with current ASA code, I found that the base NAT configuration (internet only PAT) had no bearing on the hairpin configuration. As expected, I found the same challenge around state bypass. I wanted to share a current post that demonstrates the challenges and solutions when traffic is bounced off the inside interface of the ASA.

ASA Hairping

The requirements of the configuration are as follows–

  • TestHost must be able to Telnet and Ping to Internet and PartnerHost
  • The inside interface of asav-1 must be the default gateway for TestHost
  • asav-1 is doing PAT for Internet destined traffic
  • PartnerRTR and ParnterHost have been preconfigured as shown above

The following are the base configurations for all of the devices. The configuration of asav-1 does not seem to allow communication from TestHost to PartnerHost ( network). Continue reading

Posted in Uncategorized | 3 Comments

Creating a Firepower Peer to Peer Dashboard

Peer to peer applications are a significant challenge for policy enforcement solutions. Any flows that slip through as an undetermined application type may allow the file sharing app to function. The first key to addressing this challenge is simple visibility into which hosts or users may be abusing the AUP with these types of applications. This article shows a quick and easy way to create Peer to Peer dashboard in the Firepower Management Console.

For those that have already attempted this, there are a number of challenges that may have surfaced. First there are no readily available widgets or criteria that will show the desired information. Experimenting with search constraints and the connection table quickly reveals that the desired information can be easily accessed by using the “peer to peer” Application Protocol Category search criteria. Unfortunately, the Connection Table is not readily available to the Dashboard Widgets. The Connection Summary table is available, but it does not have the Application Protocol Category (required for the search constraints).

My goal was to build a few dashboard widgets to give visibility into Peer to Peer activity. For this article I will demonstrate the steps required to build those widgets. The first widget will provide a list of Peer to Peer applications on the network, the second will provide a list of initiating IP addresses of peer to peer connections, and the third widget will sort actions taken on peer to peer traffic. All of these will be sorted based on the connection count.

CustomTablesTo overcome the challenge of not having access to the Connection Table from the Dashboard Widgets, it is necessary to create a custom table. To do so, choose Analysis, Custom > Tables, Custom Tables. 

The next step is to create a new Custom Table by choosing Create Custom Table. This allows for logical table joins between tables that contain combinable data. For this example, there is really only a need for a handful of fields from Connection Events. However, it is necessary to add at least one field from another table to allow the custom table creation. I have added the following fields from Connection Events: PeerToPeerTableApplication Protocol, Application Protocol Category, Initiator IP, and Action. I have also added IP Address from the Hosts table so it contains fields from multiple tables. This isn’t really necessary, but the UI doesn’t allow it to be saved otherwise. Continue reading

Posted in Uncategorized | Tagged | Leave a comment

Firepower Access Control Policies

The Firepower ecosystem is a powerful NGIPS/NGFW solution. At that heart of the configuration construct is what is known as the Access Control Policy. Comparing this to something familiar is possible by thinking about the much simpler filtering feature in the ASA. For comparison, an ASA’s access-list (ACL) has multiple access-control entries (ACE’s). Each of these entries can refer to object-groups, networks, and protocols and can apply a permit or deny action.

The Access Control Policy in Firepower is a similar concept, but there are many additional facets that are pulled together to provide a more comprehensive policy application mechanism. This article only covers the major areas of this policy control construct. There are items which are beyond the scope including variable sets and manipulating the behavior of http response pages.

Specific to the policy application, there are two main areas of the the Access Control Policy. The first area is what is known as Security Intelligence. In the policy, this is found on the second tab from the left and provides a framework for blacklisting. There are many feeds provided directly from Cisco’s Talos organization and are ready for consumption by the security policy.

The action for each feed that is blacklisted can be set to monitor only or to block. These events will be logged as Security Intelligence Events and may raise an Indication of Compromise. It is also this tab that will allow for the selection of a DNS Policy. Worth noting is that whitelisting is an override for blacklisting. Whitelisting does not preclude processing against the remaining Firepower Access Control Rules.

The second, and most visible, component contains the Access Control Rules. This is located in the left-most tab of the Access Control Policy. The section is a familiar construct in which rules are processed in a top-down manner until a match occurs. In the case that the action is Monitor, further processing occurs. If the action is Trust, Allow or Block, rule processing terminates and the appliance will process the traffic accordingly. A Trust action indicates that the traffic should be fully trusted and not subject to IPS checks (against the Snort signatures). Allowed traffic may be sent to IPS for additional processing.

Access Control Rule

Continue reading

Posted in Uncategorized | Tagged | 1 Comment

Saving Money with IOT Water Heater

About six months ago I installed an Energy Efficient water heater. This unit is what is known as a heat pump water heater. For those not familiar with refrigeration, this works by moving heat instead of creating heat. By contrast, traditional electric water heaters use resistance coils to heat the water. This new unit also has traditional coils that can be used for high demand or high temperature settings as well.

I guess by now everyone is wondering what this has to do with the topics we discussed at PacketU. To better understand the relationship, you can see that this Water Heater is also Connected to the Internet. The primary reasons I wanted to connect it to the Internet was to schedule the modes around my family’s usage patterns and control vacation mode from a mobile phone. When purchasing this unit I was quite skeptical and was concerned about transitioning from a simple conventional model to a mode that literally has moving parts. Continue reading

Posted in Uncategorized | Leave a comment

Firepower Indications of Compromise

Several days ago I wrote an article about Firepower Sinkhole rules. While I was confirming this in a lab, I temporarily created a custom DNS sinkhole rule. That rule classified requests for as Command and Control and returned an IP address of What I later noticed is that this caused my laptop to be classified with an IOC.

Indications of Compromise (IOCs) can be thought of as reasons why Firepower Management Console believes a host cannot be trusted or is otherwise affected by malware. These can be found in multiple places in the UI. I find the Context Explorer to be a good middle ground for most SecOps team members and a good place to notice whether current IOC’s exist.

My network is rather simple and I only currently have one IOC. In any case, I can use the Context Explorer to launch the host information for the impacted host.

IOC Context Explorer

Once the Host Profile screen is launched, I can get a little more about information about the activity that causes Firepower to believe that this is a compromised host. Continue reading

Posted in Uncategorized | Tagged | 1 Comment

Understanding ‘transport output’ and ‘access-class’

Several years ago I wrote an article called The Elusive “access-class out” CommandMy primary goal was to help CCNA students understand both the behavior of and placement of this command. My friend Anthony Sequeira done a great job in the video that is also shown in my original post. Today, I want to share another command and expand on there behavior.

For all of the demonstrations in this article, the following topology will be used. The router named iosv-2 will be the primary focus and the only place changes will be made.


Understanding Telnet:SSH Client Restrictions

Backing up for a moment, there are a couple of messages that might be displayed when an IOS device blocks outbound telnet or ssh sessions from the current exec session. These are demonstrated with a quick configuration of an transport output and access-class restriction.

//the first error is unique depending on
//if ssh or telnet is being used
iosv-2(config)line con 0
iosv-2(config-line)#transport output none
iosv-2(config-line)#do telnet
% telnet connections not permitted from this terminal
iosv-2(config-line)#do ssh -l cisco
% ssh connections not permitted from this terminal

//now we can re-enable all the protocols
//and demonstrate the other error message
iosv-2(config-line)#transport input all
iosv-2(config-line)#access-list 1 deny
iosv-2(config)line con 0
iosv-2(config)#access-class 1 out
iosv-2(config-line)#do telnet
Trying ...
% Connections to that host not permitted from this terminal

It is worth noting that in the second example, the error message is agnostic to the protocol being used (ssh or telnet). Continue reading

Posted in Uncategorized | Leave a comment

The New Strange Behavior for OSPF ‘Redistribute Subnets’

In older IOS Code, there was a specific requirement for redistributing OSPF Subnets. In almost every case, the keyword “subnets” had to be added to the redistribute command.

Older Code Example–

//notice the warning when 'subnets' is omitted
R1(config)#router ospf 1
R1(config-router)#redistribute eigrp 1
% Only classful networks will be redistributed

//and the configuration is stored exactly as it was typed
R1(config-router)#do show run | sec router
router ospf 1
 redistribute eigrp 1

//we can change the behavior by adding 'subnets'
R1(config-router)#redistribute eigrp 1 subnets

//after adding, it is stored as one would expect
R1(config-router)#do show run | sec router
router ospf 1
 redistribute eigrp 1 subnets

When I tested this in VIRL running 15.6(1S) running inside of IOS XE 3.17, the warning goes away. The question I had is whether it is still relevant or not.

Current Code Example–

//without the 'subnets' keyword
csr1000v-2(config)#router ospf 1
csr1000v-2(config-router)#redistribute eigrp 1
csr1000v-2(config-router)#do show run | sec router
router ospf 1
 redistribute eigrp 1

//with the 'subnets' keyword
csr1000v-2(config-router)#redistribute eigrp 1 subnets
csr1000v-2(config-router)#do show run | sec router
router ospf 1
 redistribute eigrp 1 subnets

I initially spent some time experimenting with this and thought that ‘subnets‘ had completely lost its relevance. However, the keyword kept popping up on its own. What I finally discovered is that IOS adds it later in in some cases. Take a close look at this strange behavior. Continue reading

Posted in Uncategorized | 3 Comments

Firepower Threat Defense — DNS Sinkholing

A few days ago I wrote an article that described Firepower DNS Policies. One item that probably warrants a little more discussion is DNS Sinkholing. Although the title of this article indicates Firepower Threat Defense, this will also work with Firepower and Firepower Services.

For this article, I would like to first share some of the challenges around getting security intelligence visibility from DNS requests. A typical enterprise environment will have an internal DNS server. So even though we know we can return “Domain Not Found” with an FTD DNS policy, that might not give us the visibility necessary to remediate a problem.

So if the host in the diagram below makes a DNS request for, what happens? Basically that request is sent to the DNS Resolver. The DNS Resolver will look to the Root Hints and eventually get the request to an Internet based DNS server that has the appropriate domain ownership. The problem with this is that the only request seen by the Firewall (FTD in our example) is the one made by the DNS Resolver. The problem here is that there is no way the Firewall can tell which host needs to be scrubbed by the SecOps team.

DNS Sinkholing

Sinkholing can solve this problem. As a DNS Reply for a known malicious request passes through FTD, it can replace the server address with one of the administrator’s choosing. This simply needs to be an address that will cause the clients to send the traffic through the Firewall. This does two things. Similar to the other DNS actions, it drastically decreases the likelihood that the host pc will connect to the malicious host on the Internet. It also allows FTD to see the subsequent connection attempts to the modified address found in the DNS Reply. Continue reading

Posted in Uncategorized | Tagged | 2 Comments

Testing the EIGRP Feasibility Condition (FC)

Last night I was going through some CCIE Routing and Switching VOD’s and found a statement I found interesting. Beyond the fact that I thought the content was far below the expert level (which is fine because a refresher or level-set is typically helpful), I believed it to be incorrect. The statement that was made is as follows:

“A neighbor meets the feasibility condition if the reported distance by the neighbor is the same as or smaller than the feasible distance of the router”

So what are my issues with this statement? First, I thought “feasible distance of the router” is ambiguous and could be assumed to be the advertised distance or the reported distance which is basically the feasible distance of the neighboring router. However, that was not my main problem with the statement. My main concern with this statement is that I have always learned that the feasibility condition is only met if the reported distance (RD) is strictly less than the feasible distance of the local route. So I set out to determine if I had a correct understanding or if the Feasibility Condition (FC) could really be met with a RD equal to the FD.

To test my theory, I built out a simple topology with a network in two different places. The second instance of is one more hop away and should work out perfectly to demonstrate this from the perspective of csr1000v-1. This should create an RD through csr1000v-2 that is equal to the local FD of csr1000v-1 for the network represented on the L1 interfaces. I also simplified the topology by removing the link between the two intermediary routers and only included transit links and the in the EIGRP topology.

Screen Shot 2016-07-04 at 9.45.49 PM

EIGRP Topology from csr1000v-1

Continue reading

Posted in Uncategorized | Leave a comment

Understanding Firepower DNS Policies

One cool feature added with Firepower version 6 is probably best described as DNS-based Security Intelligence, Inspection and Sinkholing. The thought is pretty simple. If a host issues a DNS request for a host that is known to be malicious, that response is manipulated. The manipulated response can be host not found, an alternative IP address or no response at all. This allows an administrator to provide another layer of protection by preventing hosts ready access to the IP addresses of known malicious hosts.

So the first question that might come to mind is how are hosts on the Internet classified as bad. The short answer is that Talos maintains lists of known bad fully qualified domain names (fqdn). These are actually categorized and delivered into the Firepower solution as a feed. Each of the following category can be selected into one or multiple DNS Rules.

DNS Feeds and ListsDNS Rule with Categories

  • DNS Attackers
  • DNS Bogons
  • DNS Bots
  • DNS CnC
  • DNS Dga
  • DNS Exploitkit
  • DNS Malware
  • DNS Open_proxy
  • DNS Open_relay
  • DNS Phishing
  • DNS Response
  • DNS Spam
  • DNS Suspicious
  • DNS Tor_exit_node

Continue reading

Posted in Uncategorized | Tagged | 2 Comments

Meraki MX – URL Filtering

Over the past few days, I’ve spent quite a bit of time looking at some of the advanced capabilities of modern Cisco Firewalls. My most recent testing was done with the Meraki MX 60 cloud managed Firewall product. What I have to say is this is the easiest to configure content filter I’ve ever seen. So I just wanted to take a moment and share what that looks like.Meraki MX Menu

As with all Meraki products, the MX is completely cloud managed. So to manage the device, and administrator must access the Meraki Dashboard. Once authenticated, it is simply necessary to choose Security Appliance then Content Filtering from the menu on the right.

Once on the content filtering page, the policy is self explanatory. The top section is for categories that should be blocked. While the box appears to be a free form entry field, clicking anywhere in the area presents a list of categories to choose from. The bottom section allows for manual whitelisting and blacklisting. To get a better idea on how the match is performed and the format requirements of the block criteria, the “Learn how URL blocking works” link may be selected. Continue reading

Posted in Uncategorized | 1 Comment

Be Careful with TCP Syslog and the ASA

I wanted to take just a moment to share a little gotcha that could take you by surprise. To demonstrate, I have a simple topology with an ASA in the middle. I am inspecting ICMP so ping traffic is stateful and flows properly.

TCP Syslog
To confirm connectivity,  I can ping from csr1000v-2 from csr1000v-1

csr1000v-1#ping repeat 5
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to, timeout is 2 seconds:
Success rate is 100 percent (5/5), round-trip min/avg/max = 2/6/16 ms

Now for the ASA change that can catch an administrator off guard Continue reading

Posted in Uncategorized | 2 Comments