Watchguard cloud - too much data

Abertay · February 2021

Hi,

Last couple of days i've been getting the 'too much data' error on the cloud dashboard when you try and access the exec or sec dashboards. Its only set on the default day and its never done this before.
Is there an issue with the service?

james.carson · February 2021

Hi @Abertay
There isn't a problem insofar as I'm aware. Checking my test accounts, everything seems to be fine.

If the issue persists, I'd suggest opening a support ticket under the account you're seeing that on so that our team can look into it.

Abertay · March 2021

So , i opened a case and apparently there was a change in February to limit the amount of data. Now with my timeframe set to even 1hr i can't see my top blocked clients.
This severely restricts how useful the WG cloud is. Might have to resurrect Dimension.

Bruce_Briggs · March 2021

Also, not being able to see diagnostic log messages immediately is also a negative.
Not obvious why this is the default here, since supposedly only authorized people can see the WG cloud logs.

"WatchGuard Cloud stores diagnostic log messages sent by a Firebox, but they are not visible in Log Manager or Log Search. If you need to troubleshoot an issue, you can request these diagnostic log messages from WatchGuard Technical Support."
https://www.watchguard.com/help/docs/help-center/en-US/Content/en-US/WG-Cloud/Devices/reports/log_manager_wgc.html

rv@kaufmann.dk · March 2021

Well, currently i think the Dimension log in WG Cloud is useless.

First because of the too much data issue, we cannot see debug logs without a request to WG support but also the price tag on data storage. We have logs going 6 month back which sit at 1.8TB database usage currently. At that usage consumption the price for Dimension in the Cloud would be outrageous.

Was once of the selling points not it was so much faster seaching for logs in the Cloud instead of a on-prem Dimension?

To be fair i know it takas a long time for a new product to mature when it has to replace a lot of existing features on a new platform. WG is not alone here.

Abertay · March 2021

I'm impressed you got 6mths out of Dimension. Once we hit 3mths the database runs so cripplingly slow its almost impossible to do searches.
We have to archive off data older than that to keep it usuable.

rv@kaufmann.dk · March 2021

I had support tuning the postgres not long ago due to deadlocks.

"i can tune the postgresql paramaters to let it take in more data before hitting a checkpoint. it will reduce those out of shared memory errors and allow some of the larger queries to complete. we mostly to help reports generate with large amounts of data."

and

"dimension monitoring the size of the database itself.
every few minutes dimension kicks off a query to measure the size of the database on disk. the query is serialized on date ranges which is a costly operation. it takes anywhere from 30 seconds to one minute to complete. this query is used to create the diagnostic data in the dimension ui and to display log rates over time when using the log manager function with in dimension"

I can´t find it right now, but i think support also made some changes to this on my Dimension server.

I have allocated 8GB memory and 6 vCPU.

james.carson · March 2021

We're happy to tweak settings on on-prem Dimension servers via a support case, if that's needed. Dimension's VM image is tuned to be able to run on most systems. (The vast majority of them are logging one or two firewalls on a ~20GB database partition.)

The best performance for large Dimension installs will generally come from instances that have an external database hooked up to them.

(More info here:)
https://www.watchguard.com/help/docs/help-center/en-US/Content/en-US/Fireware/dimension/database_configuration_d.html

*Note that Dimension won't move your database to the new external server, so this is usually best done on a new install.

Some tips for DImension:
-Clicking a report (like opening executive dashboard) queues and generates a on-demand report. Clicking around or clicking the same report multiple times queues multiple instances of that same report. Some reports require a lot of data to generate, and clicking the report again will make the wait longer.

-Consider scheduling time consuming reports for email or FTP delivery. Scheduling reports for the previous day at night when logging activity is at its lowest can help performance.

-Dimension prioritizes logging/ingesting log data above report generation and UI elements (this is to ensure that log data isn't missed.) Having firewalls sending debug or information level logs to a Dimension server often triples or quadruples the amount of data it has to ingest and store. If you're not actively troubleshooting, check the diagnostic log level settings in your logging settings and ensure they're set to ERROR.

KenRR · February 2023

the problem with wgc log search is still present today. i honestly dont understand how we are supposed to use the log search feature if even a single query on a single day returns errors like 'Search results are too large. Reduce the time range or enter more specific search criteria.'

how does wg recommend we use wgc log search?
has wg considered IOC checking where you may have pages of ip adresses from a third party that needs to be checked? if so is there a different solution to performing these kinds of searches?

Watchguard cloud - too much data

Comments