How many URL's Should Be Populated in Crowdstrike Category?

[jrandrusk]

We have been running the tool for a while now and have the tool configured for 10,000 entries and our ZScaler limit is 25k, but we're only seeing around 2,300 URL's listed in the Crowdstrike category. Should we be seeing more or is this normal?

[piExpr]

Default is 25k
Increased to 275k but at cost to level up your subscription
API Rate limit is 40k, I've mine set the config to 39k. When it hits this count post-processing it sleeps x mins and restarts the job..
In all honesty the update post-processing is out of sync.

[jrandrusk]

Thanks, I bumped up the limit variable from 10k to 25k, which is now giving us 6,301 URL entries in the Crowdstrike category. How many entries in your CS URL category are you seeing? Trying to see what the parity is.

[piExpr]

Are you using the latest repo? I couldn't get the latest to work. Have you looked at Zscaler audit logs with Update action, does it show success or failure?

[jrandrusk]

Just started using the latest pull yesterday and got the URL category count up to 6,300. It showed me like 15,000 rejected indicators, so that would explain the lower number.

[piExpr]

When using the latest i ended up all kinds of error with python lib and config key parser. If you don't mind sharing how's your instances setup? Is it running on containers or using some flavor of o/s with vscode?

[jrandrusk]

Is that last error line complaining about a rejected_log not existing? I ran into that and just did a ' mkdir logs/rejected_log' and re-ran the script and it started working. Probably should file an issue with that.

[piExpr]

I ended up not using the latest and couldn't figure our what all went wrong. I used the original repo now no issues. As of last sync it ran but the count of ioc is unchanged. After downloading latest fork did you have to mess with anything configuration files except config.ini?

[jrandrusk]

I just bumped the limit up to 25k since that is our max.

[jrandrusk]

Discovered that the issue seems to be that the script is doing a lookup in Zscaler to see if a given URL is categorized and if it is it rejects it, which seems to me to be backwards logic. I would expect the script to treat the Crowdstrike Threat Indicators as the source of truth versus Crowdstrike + Zscaler. To verify this just search for 'zscaler' against one of the rejected log files and you'll see. @jlangdev @tsullivan06 Can you shed any light on this?