Supervisory Analytics Tool for SOC Assessment (SAT-SA)

Background: The National Critical Information Infrastructure Protection Centre (NCIIPC) assesses the cyber resilience of Critical Sector Entities (CSEs). As part of these assessments, NCIIPC performs manual reviews of samples of security alerts and case-management records generated by Security Operations Centres (SOCs). These reviews have consistently produced valuable supervisory findings that were not evident through policies, audits, self-assessments, management reports, KPI dashboards, or compliance documentation. The purpose of these reviews is not to assess individual alerts. Rather, alert and case-management data are used as operational evidence to assess whether a CSE possesses effective capabilities relating to: (i). Threat Detection (ii). Investigation (iii).Escalation (iv). Incident Response (v).Security Operations (vi).Governance and Oversight (vii).Operational Discipline (viii).Cyber Resilience While effective, manual review is resource-intensive and difficult to scale across a growing number of CSEs and increasing volumes of security data. • Description: NCIIPC seeks a deployable Supervisory Analytics Tool for SOC Assessment (SAT-SA) that assists supervisors in analysing SOC alert and case-management data at scale.The tool should help supervisors: (i). Identify entities requiring supervisory attention. (ii).Prioritise alert samples and investigations for manual review. (iii).Detect operational weaknesses and cyber resilience concerns. (iv).Improve the efficiency, consistency and scalability of supervisory assessments. The tool is intended to support human examiners and supervisory decision-making. It is not intended to replace supervisory judgement. 1. Out of Scope The proposed solution is not intended to: (i). Function or replace as a Security Operations Centre (SOC) of CSEs. (ii).Perform real-time monitoring. (iii).Act as a SIEM platform. (iv).Act as a centralized SOC for multiple entities. (v).Continuously collect logs or telemetry from CSEs. (vi).Serve as a national cyber monitoring platform. The solution should be viewed as a supervisory analytics capability, not an operational security capability. 2. Data Environment Participants may assume access to periodic submissions including: (i). Alert metadata (ii).Case management records (iii).Investigation workflow data (iv).Escalation records (v). Alert disposition and closure information (vi). Asset and system inventory information (where available) Solutions should minimise dependence on raw logs, packet captures, customer information, or other sensitive operational data unless clearly justified. 3. Core Supervisory Problem Manual reviews often uncover weaknesses that are not visible through conventional reporting mechanisms.These weaknesses generally fall into two categories: A. Execution Gaps Situations where documented controls, governance arrangements, policies, procedures, metrics or reported capabilities suggest effective operation, but operational evidence indicates otherwise. • Examples: (i).Alerts acknowledged but not meaningfully investigated. (ii).Cases closed unusually quickly. (iii).Critical alerts closed without escalation. (iv).Repetitive or template-driven investigations. (v).Controls deployed but not effectively monitored. (vi).Operational behaviour designed to satisfy metrics without reducing risk. B. Negative Space Situations where expected evidence is absent. Examples: (i).Missing telemetry from critical systems. (ii).Absence of expected alert categories. (iii).Missing investigations or escalation records. (iv).Unexpectedly low activity levels. (v).Monitoring blind spots. (vi).Absence of evidence that would normally be expected within comparable environments. The tool should help supervisors identify both known and previously unknown indicators of these conditions. 4. Functional Requirements Data Ingestion: The tool shall: 1. Ingest structured data from multiple CSEs. 2. Support common formats such as CSV, JSON, database exports and APIs where available. 3. Support analysis of large datasets spanning multiple entities and time periods. Supervisory Analytics: The tool shall: 4. Identify indicators of detection, investigation and escalation weaknesses. 5. Detect potential execution gaps. 6. Detect potential negative space. 7. Identify anomalies, outliers and suspicious operational patterns. 8. Perform peer comparison and benchmarking across entities. 9. Generate entity-level supervisory risk indicators. 10. Prioritise entities, controls, processes and alert samples for manual review. Explainability: The tool shall: 11. Provide clear rationale for findings. 12. Present supporting evidence. 13. Support traceability and auditability of results. 14. Allow supervisors to understand why an entity or activity was flagged. • Reporting: The tool shall: 15. Generate supervisory dashboards and reports. 16. Support trend analysis across entities and time periods. 17. Enable drill-down from supervisory findings to underlying evidence. • Illustrative Supervisory Use Cases The following examples are illustrative and not exhaustive. The tool may assist in identifying: (i). High-severity alerts closed unusually quickly. (ii). Repeated alerts on the same asset without evidence of root-cause remediation. (iii). Critical alerts closed without appropriate escalation. (iv). Critical systems generating little or no security telemetry. (v). Significant deviations from peer entities. (vi).Missing monitoring coverage for critical environments. (vii).Repetitive investigation patterns suggesting superficial review. (viii).Operational behaviours that satisfy performance metrics without effectively managing cyber risk. (ix).Investigation or escalation workloads inconsistent with expected activity levels. Participants are encouraged to identify additional supervisory signals beyond these examples. 5.Deployment Requirements The solution shall operate within an NCIIPC-controlled environment. The solution must: (i).Operate in a fully offline (air-gapped) network. (ii).Require no Internet connectivity. (iii).Have no dependency on cloud services. (iv).Have no dependency on SaaS platforms. (v).Have no dependency on externally hosted AI models or APIs. (vi).Support local deployment and local data processing. Where AI or machine learning is proposed, participants shall specify: (i).Model architecture. (ii).Hardware requirements. (iii).Offline training and inference approach. (iv).Model update mechanism. (v).Explainability controls. (vi).Auditability controls. 6. Deliverables Participants should provide: (i). Solution architecture. (ii). Functional design. (iii).Analytics methodology. (iv).Data requirements. (v).Tool or Prototype (vi).Infrastructure requirements. (vii).Validation methodology. (viii).Estimated deployment and operational requirements. 7. Performance Criteria Criterion Weight. Ability to Support Supervisory Assessment, Detection of Execution Gaps, Detection of Negative Space, Explainability and Auditability, Scalability and Performance, Innovation and Additional Supervisory Insights 8. Validation Requirement Participants shall explain how the proposed solution will be validated against findings derived from expert manual review. Solutions should demonstrate their ability to identify supervisory signals, operational weaknesses and areas requiring attention with effectiveness comparable to or better than current manual sampling approaches. 9. Success Criterion A successful solution will enable NCIIPC to efficiently analyse large volumes of SOC alert and case-management data from multiple CSEs, identify entities and operational areas requiring supervisory attention, prioritise manual review effort, and preserve the quality of supervisory assurance currently obtained through expert human examination. • Expected Solution/Deliverables for Evaluation • Source Code Link (GitHub/Drive Link) • Readme with Setup Instructions • Architecture Document (Max 2 Pages) • Demo Video (Max 2 Minutes) • Technical Presentation (Max 5 Slides)