Data Models are one of the major underpinnings of Splunk’s power and flexibility.
They’re the only way to benefit from the powerful pivot command, for example.
They underlie Splunk Enterprise Security (probably the biggest “non-core” use of Splunk amongst all their customers).
Key to achieving peak performance from Splunk Data Models, though, is that they be “accelerated“.
Unfortunately (or, fortunately, if you’re administering the environment, and your users are mostly casually-experienced with Splunk), the ability to accelerate a Data Model is controlled by the extensive RBACs available in Splunk.
So what is a poor user to do if they want their Data Model to be faster (or even “complete”) when using it to power pivot tables, visualizations, etc?
This is something I’ve run into with customers who don’t want to give me higher-level permissions in their environment.
And it’s something you’re likely to run into – if you’re not a “privileged user”.
Let’s say you have a Data Model that’s looking at firewall logs (cisco ios syslog). Say you want to look at these logs going back over days or weeks, and display results in a pivot table.
If you’re in an environment like I was working in recently, where looking at even 100 hours (slightly over 4 days) worth of these events can take 6 or 8 or even 10 minutes to plow through before your pivot can start working (and, therefore, before the dashboard you’re trying to review is fully-loaded).
Oh!
One more thing.
That search that’s powering your Data Model? Sometimes (for unknown reasons (that I don’t have the time to fully ferret-out)), it will fail to return “complete” results (vs running it in Search).
So what is a poor user to do?
Here’s what I’ve done a few times.
I schedule the search to run every X often (maybe every 4 or 12 hours) via a scheduled Report.
And I have the search do an outputlookup to a CSV file.
Then in my Data Model, instead of running the “raw search”, I’ll do the following:
| inputlookup <name-of-generated-csv>
That’s it.
That’s my secret.
When your permissions won’t let you do “what you want” … pretend you’re Life in Ian Malcom‘s mind – find a way!