Tag: Security

So the last few blogs posts have been really exciting and enlightening with a clear path for migration, unfortunately, we hit the part where things get a bit bleak in the migration path.

Users and permissions are very rich and granular in Splunk. The concept of namespaces are innovative and very useful when you have to carve out sub-administration tasks and data targeting.

There is no concept of namespaces in Azure Sentinel. There is no wide-spread ability to “hide” things with permissions in Sentinel. You can see some of the limited actions/roles that are available in the article Permissions in Azure Sentinel. That being said, there are some things that can be done, but they are really not a matching one to one of the security features in Splunk.

What might we want to carve out permission-wise? Here’s a simple list:

1. Queries
2. Lookups
3. Alerts / Incidents
4. Dashboards
5. Tables / Schemas

Let’s take a look at each of these.

Query Permissions

Queries are stored in various ways in Azure Sentinel. Since Log Analytics is the backing for Azure Sentinel, most of them will be saved into Log Analytics. When being stored, they can be simply stored directly via Log Analytics or have various extra properties that define them as special for display in Sentinel. Others could be stored as sub-objects attributes.

Basically there is no concept of pointers to a query like in Splunk. Its a copy by value type of thing. Queries can be copied/used across Analytic rules and Hunting queries. They can also be saved in Log Analytics for re-use but only at the Log Analytics level. Queries saved as functions can be called at higher levels such as Sentinel.

Hunting queries and Analytics Rules can’t be targeted at a user or group level. If you have access to Sentinel, you have access to all the hunting queries. If you have queries that are targeted at a user / namespace in Splunk, they will become global in a single instance of Azure Sentinel. And since Azure Sentinel can’t be pointed at multiple log analytics workspaces, you can’t really define things at a Azure Sentinel level that will then span across all Log Analytics workspaces.

At a Log Analytics level, you can save queries to the workspace. Again, there is no user / group targeting here. If you have access to the Log Analytics workspace, you have access to all saved queries.

Lookup Permissions

As you read about in a previous blog in this series, there are several ways to create lookups. Some of these can content sensitive data so having a way to lock them down is going to be important. Remember, the three types of lookups are:

1. Log Analytics Table: As you will see later, log analytics tables can be targeted using custom roles.
2. Watchlists: Global for the Sentinel instance, everyone can see them.
3. Azure Storage: Using Azure Storage means you would have to have knowledge of the file location and consequently access to the file (via SAS). Problem is, it is easy to look across all queries and be able to find any references to these external files.

The only real option you have for lockdown for lookups is via custom Log Analytics Table imports.

Alerts / Incidents

If you have access to the Azure Sentinel instance, you will see all incidents. There is currently no way to filter or hide incidents in Sentinel based on tagging or any other means. A SOC analyst will be able to see all alerts and incidents.

Dashboards

Since Dashboards are an Azure Resource that show up in the Resource Group, you can set IAM on them such that individuals won’t even be able to see them when they navigate to the Azure Portal. This simply prevents them from seeing a pre-created dashboard. If they were to figure out what a query was that showed data they shouldn’t see and build their own dashboard (provided they have Contribute access to Azure resource group), they could gain access to data they should probably not be looking at.

Tables / Schemas

Tables and data are stored in Log Analytics. As such, if you want to hide tables, you will need to utilize Azure Custom Roles with predefined table sets. You can reference how to this in the article Manage access to log data and workspaces in Azure Monitor. Doing this manually could be a months long process just by itself.

The way to make this work, is to enumerate the namespace in Splunk and then create custom roles for each of the namespaces and the tables they have access too. This will only allow you to limit access to the underlying tables, not any other elements in Azure Sentinel. However, by the nature of its function, it would prevent a person from being able to execute Hunting queries or to view Dashboards that reference those tables since they don’t have access to them.

Users

Users can be exported from the Splunk instance. If they have their email property setup and they exist in the target Azure AD (which could be automated to provision in the case they are not), then they can be added with the basic permissions of Owner, Contributor and Reader in Log Analytics or Sentinel. Anything customized will need to be via custom roles after the user has been created.

If you do go down the custom roles based on tables (which are likely based on namespaces), you will need a mapping file from the Splunk namespace to the role name in Sentinel.

Resolving the Permission problem.

As you can see, there really is no real high-level concept of permissions in Azure Sentinel and Log Analytics. The lowest level you can set permissions is in the Log Analytics workspace via IAM/Custom roles for tables and then hope everything works right in the higher layers.

Logging Queries

If you choose to migrate without considering any permissions during the migration, it is probably going to be a good idea to log and monitor any queries that are being sent to the Log Analytics workspace (directly or via Sentinel). If there are very sensitive tables contained in the workspace and folks are running queries that should not be doing so, you should be alerted to that (from another workspace/Sentinel?)

So, using Sentinel to monitor Sentinel becomes a thing. Isn’t recursion great?

Summary

Net net…Azure Sentinel is missing some pretty substantial permission based features that you enjoy in other SIEMs. It is up to you to determine if the functionality you are gaining, out weighs the short term drawbacks of losing out on these important security features.

Splunk to Sentinel Blog Series

1. Part I – SPL to KQL, Exporting Objects
2. Part II – Alerts and Alert Actions
3. Part III – Lookups, Source Types and Indexes
4. Part IV – Searches
5. Part V – Dashboards and Reports
6. Part VI – Users and Permissions
7. Part VII – Apps

References:

1. My Twitter
2. My LinkedIn
3. My Email

In Splunk, searches are saved queries. These queries can be used in any number of places, or simply just run when needed. Queries are built in the Splunk Programming Language (SPL).

In Part I of this series we discusses exporting the objects in the Splunk instance. Search queries you will discover are at the center of everything.

We have already discussed Alerts and how they use queries, but as we will see in the next post in this series other things like Dashboards and Reports also use them.

Searches

As we discusses in Part I of this series, we have to export the objects in Splunk as the first step. One of the things you will realize is that “Searches” is an overloaded term. In order to get dashboards and reports out, you have to hit the “searches” REST endpoint.

$url = "$apiUrl/servicesNS/-/search/saved/searches..."

When these object are exported, there are a few properties you will need to interrogate to determine what the search was created for:

1. Report = {entry}.content.is_scheduled
2. Dashboard = {entry}.content.isdashboard

We will explore these instances in the next series as they have other higher level components that must be considered, for the purposes of this blog, we simply want to discuss where are the possible places we can “save” these queries in Azure Sentinel.

1. Sentinel Hunting Queries

2. Sentinel Analytics (Scheduled Rule)

3. Log Analytics (Saved Query)

Each of the items above have a different endpoint to create them:

#Hunting Query (Sentinel)
$url = "https://management.azure.com/subscriptions/$subscriptionId/resourceGroups/$resourceGroupName/providers/Microsoft.OperationalInsights/workspaces/$workspaceName/savedSearches/$($id)?api-version=2017-04-26-preview"
#Scheduled Rule (Sentinel)
$url = "https://management.azure.com/subscriptions/$subscriptionId/resourceGroups/$resourceGroupName/providers/Microsoft.OperationalInsights/workspaces/$workspaceName/providers/Microsoft.SecurityInsights/alertRules/$($ruleId)?api-version=2021-03-01-preview";
#Saved Query (Log Analytics)
$url = "https://management.azure.com/subscriptions/$subscriptionId/resourceGroups/$resourceGroupName/providers/microsoft.insights/scheduledqueryrules/$($name)?api-version=2018-04-16";

Each of these endpoints require a POST with very specific properties to tell Sentinel or Log Analytics what to create and what it is for. For example, Azure Sentinel Hunting queries can be targeted at MITRE Framework Tactics, so you’d include those, however, Splunk has no concept of MITRE that you can pull from so you’d have to add that from a reference file.

As was discussed in the first post in this blog series, the above items are the easy part, its the tokenization of the Splunk query and then conversion to KQL that is the hard part.

Splunk to Sentinel Blog Series

1. Part I – SPL to KQL, Exporting Objects
2. Part II – Alerts and Alert Actions
3. Part III – Lookups, Source Types and Indexes
4. Part IV – Searches
5. Part V – Dashboards and Reports
6. Part VI – Users and Permissions
7. Part VII – Apps

References:

1. My Twitter
2. My LinkedIn
3. My Email

In the previous two blog series articles, we explored exporting objects, alerts and alert actions. We also looked at the concept of converting SPL to KQL. As part of that conversion process their are some special things that will need to be present in order for the queries to run properly on either the Splunk or the Azure Sentinel side.

These items include lookups, source types and indexes.

Lookups (Splunk)

Lookups are an important part of any SIEM/SOAR system. Typically these make up special lists of users, or more commonly Indicators of Compromise such as IP Addresses, Domain Names, etc.

When it comes to Splunk, you can do lookups in a number of different ways. This includes a file upload with can be either a plaintext CSV file, a gzipped CSV file, or a KMZ/KML file.
The maximum file size that can be uploaded through the browser is 500MB.

You can also do script/command based lookups:

So how do lookups translate into Azure Sentinel? There are several ways:

1. Log Analytics Table
2. Watchlists
3. Azure Storage

Picking between the three options depend on the size and use of the actual lookup table. Note, as we will see in a later post, Sentinel doesn’t have a very robust permissions scheme (yet), so they is something to consider as well. In my opinion, this is probably the largest factor when considering moving to Azure Sentinel from another SIEM.

Log Analytics Table

You can use the Data Collector API to bring in Lookups into Azure Sentinel / Log Analytics.

https://docs.microsoft.com/en-us/azure/azure-monitor/logs/data-collector-api

Once imported, they would then show up like this:

Watchlists

Watchlists are the almost identical mapping of a Lookup in Splunk. As you can see here, I imported the exported CSV file programmatically from Splunk into a Watchlist in Azure Sentinel:

Azure Storage

Kusto Query Language (KQL) allows for the runtime creation of a table based on an external file. In this case, we can export the CSV files from Splunk into Azure Storage and then use KQL to bring it into the queries:

https://docs.microsoft.com/en-us/azure/data-explorer/kusto/query/externaldata-operator?pivots=azuredataexplorer

Source Types

In addition to lookup data, a vital component is you have to have log data to run your queries on. This log data comes in all kinds of shapes and formats. There are about 150 out of the box Source Types in Splunk.

Source types defined how data is formatted such that the indexers can index it. It is very much an ingest time type of thing.

Indexes

You can consider these like tables in a database. They have a name, rows and columns with type. Splunk stores the data from logs in an “index”. This data is ingested using the source types mentioned above.

In order to determine if a query is going to run, you have to gain access to all the indexes, then export the schema of those indexes. This of course would all go into the tokenizer / compiler and then save the graph of the query for comparison to the “tables” in Azure Sentinel.

As you can imagine, it is not going to be a one to one mapping on the other side. Some things are pretty common, others, not so much.

As we will see in Part VII, Apps have the ability to import new source types, which then in turn create new indexes. As we will see, Apps hold some similarity to “Connectors” in Azure Sentinel.

Ultimately, the goal is to ensure that we have the same Source Type -> Data Source mappings such that the queries are converted properly. In many cases, you’ll need to create a mapping file of the properties on the left side of the equation with the properties on the right side of the migration equation.

Splunk to Sentinel Blog Series

1. Part I – SPL to KQL, Exporting Objects
2. Part II – Alerts and Alert Actions
3. Part III – Lookups, Source Types and Indexes
4. Part IV – Searches
5. Part V – Dashboards and Reports
6. Part VI – Users and Permissions
7. Part VII – Apps

References:

1. My Twitter
2. My LinkedIn
3. My Email

Continuing where we left off in Part I, I will explore how to convert Splunk Alerts and Alert Actions into the Sentinel equivalents.

Alerts

Alerts have the same core problem as everything else in Splunk. They are built on SPL queries and must be converted in order for anything meaningful to be accomplished. This means we have to export the alert object, grab the query, convert it, and finally check that it actually executed in Azure Sentinel and then perform the migration.

Alert Actions

Once you have the alert query converted, the Alert very likely has an action tied to it. Splunk provides several out of the box actions:

Migrating these can be a daunting task if you are doing it manually. Not something I’d suggest trying to do in a large Splunk instance. The first step is to export the alert actions into their JSON form:

And then interrogate the various properties for the Alert Action.

Now that you know how to convert the Alert Actions, you need to find all the alerts that are using the actions so you can create them on the Azure side.

Alerts

Alerts are queries that when the query hits a set of targets, will execute one or more Alert Actions. The first step to migrating these is to export them.

Reviewing the exported JSON object, you will find several interesting properties.

The “QualifiedSearch” and “Search” property is the query that the alert is based off of.

This “Actions” property will tell you what actions have been activated for that alert query.

Once you have the list of activated actions, you have to figure out what information you will need to create the corresponding item in Azure Sentinel or Log Analytics.

In the example above, the logevent action has been enabled for the alert. Review the remainder of the properties, you will find a serialization technique Action.{ActionName}.{PropertyName}.{SubPropertyName0}….

Once you get the hang of it, you can then take the exported data and start to build out the migration tasks that need to occur for each Alert Action Type. Here is a basic mapping based on the items above:

1. Email -> Action Group
2. LogEvent -> Log to Log Analytics
3. Lookup -> Insert into Lookup (more to come on lookup migration later in this series)
4. SMS -> Action Group
5. Webhook -> Action Group
6. Script -> Runbook
7. Custom / App -> Logic App / Runbook

Everything above can be accomplished using Azure Management REST queries (I know as I have done the mappings successfully). Some of these have corresponding Azure CLI or PowerShell commands, however some of them do not. So the best approach is to implement everything using simple REST calls.

Splunk to Sentinel Blog Series

1. Part I – SPL to KQL, Exporting Objects
2. Part II – Alerts and Alert Actions
3. Part III – Lookups, Source Types and Indexes
4. Part IV – Searches
5. Part V – Dashboards and Reports
6. Part VI – Users and Permissions
7. Part VII – Apps

References:

1. My twitter
2. My LinkedIn
3. My Email