NTP: Server Unsynchronized on HP1820 Switch

xxup

T15 with 12.5.5 B630561

I need to be ready for the eventual release of the fixed 12.5.5 and the 12.6.2 upgrades, so I have restarted the Test Lab using the T15. The T15 has been upgraded to the latest build available on WatchGuard Cloud.

The Firebox, like all my Fireboxes, connects to the Australian NTP Servers at 0.au.pool.ntp.org, 1.au.pool.ntp.org and 2.au.pool.ntp.org. The Firebox is also set up to be the NTP server. There is a policy that allows any trusted to access the Firebox on port 123. I can see log messages on the Traffic Monitor that confirm the devices are able to connect to the Firebox on port 123.

For Firebox testing I use a simple HPE 1820-8G-POE+ (65W) switch running the latest firmware (v2.09). The NUC-based Windows 2012R2 Server is connected to it and I can hang WatchGuard Access Points off it to do GWC testing etc. Great little box.

Since restarting the Lab, I notice that the HP switch is coming up with "Server Unsynchronized" message on the SNTP page. A review of the manual suggests that there is possibly something wrong with the Firebox connection to its peers. Aparently, it gets this from the leap indicator field in the SNTP message.

I know that last time I used the lab the switch was synchonising correctly, but that was a previous firmware release on both the Firebox and the Switch.

Any thoughts?

Bruce_Briggs

The docs don't mention SNTP, so perhaps XTM doesn't really support it?
Or perhaps it is yet another bug in V12.5.5/12.6.2 U1

xxup

You may be right. I have turned on logging of traffic from the Firebox, then rebooted the firebox and checked the Dimension reports and there is no port 123 traffic from the firebox to external NTP servers.. There is lots of other traffic (ports 53, 80, 443, 10108 etc), but no port 123..

I need to get another Firebox and check it out too.. Sigh!

Bruce_Briggs

I just turned on logging for packets from the firewall, and I don't see any NTP packets being logged either.
And NTP is enabled on my T20w 12.6.2 U2

Bruce_Briggs

My "Any from firebox" policy is also not logging anything for packets going to WG Cloud.
It looks like "Any from firebox" does not log everything coming from the firewall.
Perhaps it is not logging NTP.
A TCPDUMP should show if that is true or not.

xxup

Hmmm. Looks like time to drag out the old Wireshark software and see what is going on between the Firebox an the NTP Pool and between the Firebox and the switch.. Oh joy!

xxup

The switch is now using the Firebox NTP server.. There was 377 requests with 147 failures.. My guess is that it happened about two hours after I set it up initially. . Perhaps, I just need to be more patient?

greggmh123

@xxup said:
The switch is now using the Firebox NTP server.. There was 377 requests with 147 failures.. My guess is that it happened about two hours after I set it up initially. . Perhaps, I just need to be more patient?

Patience is overrated! One of my favorite comic strips is of two vultures on a tree branch. One of them says to the other, "Patience my a$$! I'm going to go kill something!"

xxup

**Some more information. **

Another review of yesterday's logs from the Firebox show that any-from-firebox logging was turned on at 2020-10-20 11:09:46 AEST and the Firebox was rebooted shortly after that time. However, the first log entry to port 123 was:
2020-10-20 12:32:10 Allowed 203.0.113.26 220.158.215.21 123 20,140 1

Which was the Firebox talking to 2.au.pool.ntp.org.

My experience with booting UNIX servers is that they check their time via NTP during the boot process (toward the end). But the Firebox did not send a request for a time check for almost 90 minutes after the reboot.

Do others think that this might be a bug?

Bruce_Briggs

Windows default is 1 hour for domain computers
For non-domain computers, the default interval is 7 days supposedly.

If the current time is quite accurate as compared to the last NTP check, then a relatively long time until the next check is not unreasonable.

xxup

Except that modern switches are able to determine if the NTP server source has been synchronised, and will refuse to accept them as a time source. The consequence is that the switch (or any other device without some kind of battery backup) remains on 1970 time.

Bruce_Briggs

1970 was a good year, at least for me

xxup

Summer of '69 for me..

greggmh123

My Cisco SG300-28 and SG300-10PP both talk to my firewall (currently a T20) for SNTP and get the correct time. All I need to do to see the traffic is to change and re-apply a setting on either switch. I used to have an HP 1820-something and it worked as well.

Bruce_Briggs

isn't that a song?

xxup

Where do you think the idea came from?

xxup

@Greggmh123 said:
My Cisco SG300-28 and SG300-10PP both talk to my firewall (currently a T20) for SNTP and get the correct time. All I need to do to see the traffic is to change and re-apply a setting on either switch. I used to have an HP 1820-something and it worked as well.

Yep.. I get that. However, in this case we have a Firebox that has woken up after a long sleep (e.g. it could also be a brand new box.) and a switch that is trying to set time through NTP and realising that the Firebox (as the NTP server) has not attempted to set its own time, and won't for at least another 90 minutes. Throw an Windows AD Server in there and things can get exiciting.. AD Servers do not like having their time messed up (I need to find this reference in my AD book)..

greggmh123

@xxup said:

@Greggmh123 said:
My Cisco SG300-28 and SG300-10PP both talk to my firewall (currently a T20) for SNTP and get the correct time. All I need to do to see the traffic is to change and re-apply a setting on either switch. I used to have an HP 1820-something and it worked as well.

Yep.. I get that. However, in this case we have a Firebox that has woken up after a long sleep (e.g. it could also be a brand new box.) and a switch that is trying to set time through NTP and realising that the Firebox (as the NTP server) has not attempted to set its own time, and won't for at least another 90 minutes. Throw an Windows AD Server in there and things can get exiciting.. AD Servers do not like having their time messed up (I need to find this reference in my AD book)..

I'll have to swap my T20 for my T35 to see how long it takes to get NTP synced. I have my AD servers go to us.pool.ntp.org directly vs. using a Firebox for their NTP. I do that for two reasons: one is to prevent this very issue, the other is for consistency across clients, most of whom do not have a Firebox.

xxup

I might have to reconsider the design. It made sense to have a single time source on the local network. The Firebox is logical- if it is down there is no Internet anyway. I didn't want our 100mbps bandwidth cluttered with NTP traffic from every device on site and these days, even for a small business it adds up very quickly..

greggmh123

@xxup said:
I might have to reconsider the design. It made sense to have a single time source on the local network. The Firebox is logical- if it is down there is no Internet anyway. I didn't want our 100mbps bandwidth cluttered with NTP traffic from every device on site and these days, even for a small business it adds up very quickly..

While "the Firebox is logical- if it is down there is no Internet anyway" is true, if it is WRONG then you risk messing up AD and any other device looking to it for time. If your AD server uses an external NTP server, then it really doesn't care if the firewall is down for a while.

When I said that "I have my AD servers go to us.pool.ntp.org directly vs. using a Firebox for their NTP", I should have clarified that everything else (switches, cameras, etc.) is allowed to hit the Firebox for NTP, but not allowed to go public for NTP. I cannot imagine that NTP's UDP traffic could add up to much of anything, but I also haven't looked up what that traffic amount is.