Add Web Session summarization playbook using log ingestion API - #14976
Open
v-sabiraj wants to merge 3 commits into
Open
Add Web Session summarization playbook using log ingestion API#14976v-sabiraj wants to merge 3 commits into
v-sabiraj wants to merge 3 commits into
Conversation
Deploy an hourly managed-identity playbook that summarizes Web Session data into custom Log Analytics tables through a data collection endpoint and rule.
Contributor
There was a problem hiding this comment.
Pull request overview
Note
Copilot was unable to run its full agentic suite in this review.
Adds an Azure Resource Manager (ARM) template to deploy an hourly Logic Apps playbook that summarizes Web Session logs and ingests results into custom Log Analytics tables via DCE/DCR log ingestion.
Changes:
- Adds four custom Log Analytics tables for summarized Web Session data.
- Adds a Data Collection Endpoint (DCE) + Data Collection Rule (DCR) with transformations to map incoming payloads into those tables.
- Adds a managed-identity Logic App workflow that queries Web Session data hourly and posts chunked payloads to the ingestion endpoint.
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
| }, | ||
| "type": "ApiConnection", | ||
| "inputs": { | ||
| "body": "let nowTime = todatetime(\"@{variables('nowTime')}\");\nlet lastRecievedTime = toscalar (\n union isfuzzy=true\n (\n WebSession_Summarized_ThreatInfo_CL \n | summarize max(EventTime_t)\n | extend max_TimeGenerated = datetime_add('hour',1,bin(max_EventTime_t,1h))\n ),\n (\n print (nowTime)\n | extend max_TimeGenerated = datetime_add('day',-1,bin(print_0,1h)) \n | project-away print_0\n )\n | summarize max(max_TimeGenerated)\n );\n_Im_WebSession(starttime=lastRecievedTime,endtime=bin(nowTime,1h))\n| where TimeGenerated \u003c bin(nowTime,1h)\n| where (ThreatName != 'None' and isnotempty(ThreatName)) or (ThreatCategory != 'None' and isnotempty(ThreatCategory)) or ThreatRiskLevel \u003e 60 or toint(ThreatOriginalConfidence) \u003e 0 or EventSeverity in ('Medium','High') or isnotempty(ThreatField)\n| summarize EventCount=count() by ThreatName, ThreatCategory, ThreatRiskLevel, toint(ThreatOriginalConfidence), EventSeverity, ThreatField, SrcIpAddr, SrcUsername, DestDomain = tostring(parse_url(Url)[\"Host\"]), bin(TimeGenerated, 1h), EventResult, DstIpAddr\n| extend EventTime = TimeGenerated", |
| }, | ||
| "type": "ApiConnection", | ||
| "inputs": { | ||
| "body": "let nowTime = todatetime(\"@{variables('nowTime')}\");\nlet lastRecievedTime = toscalar (\n union isfuzzy=true\n (\n WebSession_Summarized_SrcInfo_CL \n | summarize max(EventTime_t)\n | extend max_TimeGenerated = datetime_add('hour',1,bin(max_EventTime_t,1h))\n ),\n (\n print (nowTime)\n | extend max_TimeGenerated = datetime_add('day',-1,bin(print_0,1h)) \n | project-away print_0\n )\n | summarize max(max_TimeGenerated)\n );\n_Im_WebSession(starttime=lastRecievedTime,endtime=bin(nowTime,1h))\n| where TimeGenerated \u003c bin(nowTime,1h)\n| summarize SrcBytes = sum(SrcBytes), DstBytes = sum(DstBytes), EventCount = count() by HttpUserAgent, EventResultDetails, EventResult, UrlCategory, NetworkApplicationProtocol, HttpRequestMethod, HttpContentType, EventTime = bin(TimeGenerated, 1h), EventProduct, EventVendor, EventType, bin(TimeGenerated,1h)\n| extend EventTime = TimeGenerated", |
| "kind": "Linux", | ||
| "properties": { | ||
| "description": "Data collection endpoint for Web Session Essentials summarized logs", | ||
| "networkAcls": { "publicNetworkAccess": "Enabled" } |
| }, | ||
| "type": "ApiConnection", | ||
| "inputs": { | ||
| "body": "let nowTime = todatetime(\"@{variables('nowTime')}\");\nlet lastRecievedTime = toscalar (\n union isfuzzy=true\n (\n WebSession_Summarized_DstIP_CL \n | summarize max(EventTime_t)\n | extend max_TimeGenerated = datetime_add('hour',1,bin(max_EventTime_t,1h))\n ),\n (\n print (nowTime)\n | extend max_TimeGenerated = datetime_add('day',-1,bin(print_0,1h)) \n | project-away print_0\n )\n | summarize max(max_TimeGenerated)\n );\n_Im_WebSession(starttime=lastRecievedTime,endtime=bin(nowTime,1h))\n| where TimeGenerated \u003c bin(nowTime,1h)\n| summarize SrcBytes = sum(SrcBytes), DstBytes = sum(DstBytes), EventCount = count() by DstIpAddr, SrcIPIsPrivate = ipv4_is_private(SrcIpAddr), DstPortNumber, DestDomain = tostring(parse_url(Url)[\"Host\"]), DstHostname, EventResult, EventResultDetails, EventProduct, EventType, bin(TimeGenerated,1h)\n| extend EventTime = TimeGenerated", |
| ] | ||
| } | ||
| }, | ||
| "Initialize_variable_-_TotalCharaterLengthDstIP": { |
| @@ -0,0 +1,780 @@ | |||
| { | |||
Renames the custom summarization tables and DCR streams to their V1 variants (WebSession_Summarized_SrcInfoV1_CL, SrcIPV1_CL, DstIPV1_CL, ThreatInfoV1_CL) to avoid conflicting with the existing tables created by the previous playbook version. Also: - Adds an explicit TimeGenerated column to each table schema and DCR transform. - Updates the playbook default name to SummarizeWebSessionData-logingestion and the title to clarify it uses the Log Ingestion API. - Updates the lookback KQL queries to read from the new V1 tables.
Documented the new `SummarizeWebSessionData_logingestion` playbook with summary, deployment steps, required parameters, and post-deployment API authorization guidance for Logs Ingestion API-based ingestion. Also corrected the Azure Gov deploy badge URL in the existing `SummarizeWebSessionData` README.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Deploy an hourly managed-identity playbook that summarizes Web Session data into custom Log Analytics tables through a data collection endpoint and rule.
Required items, please complete
Change(s):
Reason for Change(s):
Version Updated:
Testing Completed:
Checked that the validations are passing and have addressed any issues that are present:
Guidance <- remove section before submitting
Before submitting this PR please ensure that you have read the following sections and filled out the changes, reason for change and testing complete sections:
Thank you for your contribution to the Microsoft Sentinel Github repo.
Change(s):
Reason for Change(s):
Version updated:
Testing Completed:
Note: If updating a detection, you must update the version field.
Checked that the validations are passing and have addressed any issues that are present:
Note: Let us know if you have tried fixing the validation error and need help.