Syslog traffic issues on Firebox
Syslog traffic from a T80 not making it over the tunnel to the Syslog server
-HQ has a Syslog server @ 10.1.11.116. Sitting behind an M4800 firebox
-Branch office firewall is a T80 running 12.8.0, with SYSLOG pointed to 10.1.11.116
-Policy based VPN (not virtual interface VPN) is set up between the fireboxes and working properly
-Syslog server does NOT see capture any logs from the branch firewall
-From the T80 I can view Traffic Monitor and observe all normal traffic getting logged here
-Dimension logging in the cloud is receiving logs and recording them for the branch office
-I have “Enable logging for traffic sent from this device” checked under the Diagnostic Logs settings
- Why do I never see Syslog traffic in Traffic monitor? This would be useful for troubleshooting. Shouldn't this be seen?
- What interface should I expect the source of the Syslog traffic to be coming from, since it's generated on the firebox?
When viewing PCAPs from the T80's physical LAN interfaces, I never see Syslog traffic.
When I view a PCAP the T80's WAN interface I see my Syslog traffic with the branch offices PUBLIC WAN IP as the source, Syslog server's private address (10.1.11.116) as the destination, and the destination interface is the WAN interface, not a tunnel. Assuming I am onto something here.
- Is it not possible to use a Syslog server that over a point to point VPN? Would route based VPN work better?
Let me cut to the chase. I have other branch offices set up seemingly the exact same way and Syslog traffic makes it to the Syslog server. I do still see some oddities from these working set ups:
-Traffic Monitor from these working branch still do not catch the Syslog traffic, even though Syslog traffic does make it over the tunnel and the Syslog server receives it.
-I still do not see Syslog traffic in PCAP from the working firebox's. Tried all and I can never see it from the branch.
When reviewing the working setup, the syslog server sees the Syslog traffic come from the branch offices VLAN1 interface IP
- How is this chosen? Is there a precedent of using the lowest VLAN interface or the interface with the lowest IP address or something? Curious about this, because the branch office with the problem does not use VLAN interfaces. It just has a trusted interface.
Of course I am going to open a ticket as well, but all recent cases have taken DAYS to get a response so asking here too.
Comments
Hi @MGNL
I was able to find your case -- it appears to have been created this morning. I'll get it over to the team that can assist with this issue.
-Traffic from the firebox is logged via a hidden rule called "any-from-firebox." If you'd like to see logging for this you'll need to turn it on. (It sounds like you've already done this.) The firewall will only log the traffic if it's actually sent -- so if the traffic is failing because it can't ARP for the address it's sending to on that interface, it'll just drop as a network failure. Logs always show complete connections.
The firewall is most likely sourcing the syslog traffic from one of its external interfaces. If that's the case you'll need to make a rule on your firewall to apply the NAT options you want to use (likely writing the IP as something internal so it can go across the tunnel as you expect.)
See
(About Policies for Firebox-Generated Traffic)
https://www.watchguard.com/help/docs/help-center/en-US/Content/en-US/Fireware/policies/policies_firebox_generated_traffic_about.html
Jump down to the section about BOVPN and Set source IP addresses.
This article goes over most of your other questions as well.
I would suggest:
-Use the above article to expose the "any from firebox" rule.
-Create a new syslog packet filter from "Firebox" to the syslog server's IP.
-In the advanced -> NAT section of the policy, set the source IP as the firewall's IP on the network you want it to exit on. Use Dynamic NAT -> All Traffic in this policy -> Set source IP, and define the IP.
-Move the new policy above the any-from-firebox rule.
Whenever you make rules that go above the any-from-firebox rule, always make them as specific as possible to ensure that you don't accidentally catch traffic you didn't intend to catch in that rule.
To answer your questions:
1.Why do I never see Syslog traffic in Traffic monitor? This would be useful for troubleshooting. Shouldn't this be seen?
See above, the connection is likely not completing. Incomplete connections that are failing can be investigated by using TCPDUMP in the diagnostic options of the firewall.
If the destination IP is not on a network owned by the firewall, it will usually use the lowest numbered external interface to send that traffic. This can be modified by SD-WAN or PBR in rules, so this is not always the case, though.
It is possible with either -- an appropriate rule to send it where you want to send it is sometimes needed. With a normal BOVPN, you also have the option of making a tunnel from the external IP of the firewall to the internal IP of the syslog server if you want to preserve that IP for whatever reason.
If the destination matches the network the firebox owns, it will use that. If it does not, it will usually use the lowest numbered external interface, with some exceptions. The article I linked further up explains some of these.
-James Carson
WatchGuard Customer Support
You're the man. New ACL with DNAT did fix the issue. Thanks. Closing the ticket