Platform Explore docs ↓

Build on the
On-CallSRE Platform

Use our autonomous multi-agent system to create proactive, self-healing infrastructure with the most advanced AI models on the market.

Start exploring See developer docs

The 2 AM Page.

It starts with a Slack ping. A critical service has degraded, and an engineer is woken up.

Modern microservice architectures are incredibly powerful, but when things go wrong, they are notoriously difficult to debug.

An engineer logs on at 2:00 AM. They are tired. They face hundreds of cascading alerts, noisy logs, and complex interdependent systems. They have to jump between Prometheus, Grafana, GitLab, and terminal windows just to form a basic hypothesis of what went wrong.

Mean Time To Resolution (MTTR) stretches into hours. Burnout increases. Engineering velocity slows down.

We built Always-On-CallSRE to change this story.

Instead of waking up an engineer, an autonomous multi-agent system intercepts the alert. It pulls the metrics, analyzes the code history, forms a root cause hypothesis, writes the patch, validates it against security policies, and opens a Merge Request. By the time the engineer wakes up, the fix is already waiting for their approval.

Hierarchical Orchestration

Follow the incident lifecycle as the Orchestrator Agent delegates tasks sequentially and branches out to specialized subagents for deep analysis and deployment.

Observability / Webhook / Scheduler
Orchestrator Agent root_agent.py
Alert Intake Agent
Observability Agent
Root Cause Analysis Agent
Code Analysis Subagent
Infrastructure Subagent
Synthesized RCA Hypothesis
Fix Generation Agent
Validation Agent
Deployment Agent
GitLab MCP Server
Production Merge Request
Postmortem Agent

The Agent Roster

Each component operates autonomously with Gemini 2.5 Pro, possessing strict system prompts tailored for their specific role in the SRE lifecycle.

0

Proactive Monitoring

agents/monitoring_agent.py

Runs every 10 minutes. Fetches telemetry and asks Gemini for leading failure indicators. Triggers workflows before systems go down.

1

Alert Intake

agents/alert_agent.py

Parses incoming webhooks (GitLab pipelines, Prometheus alerts). Classifies severity and structures alerts for downstream processing.

2

Observability Agent

agents/observability_agent.py

Interfaces with the MCP Server to pull metrics and Grafana dashboards. Injects simulated context (memory leaks, CPU spikes) for demo fallback.

3

RCA Manager Agent

agents/rca_agent.py

Assesses incident complexity. Delegates deep-dives to Code Analysis and Infrastructure subagents, then synthesizes their findings into a final hypothesis.

4

Fix Generation Agent

agents/fix_agent.py

Writes production-grade code patches using Gemini based on the RCA hypothesis and the pulled codebase context.

5

Validation Agent

agents/validation_agent.py

Runs simulated DevSecOps checks: SAST, dependency scanning, unit test validation, and code style compliance to ensure patch safety.

6

Deployment Agent

agents/deployment_agent.py

Interfaces with the GitLab MCP Server to automatically create a Merge Request containing the validated patch and RCA summary.

7

Postmortem Agent

agents/postmortem_agent.py

Generates a comprehensive markdown incident postmortem including: timeline of events, root cause analysis, fix applied, blast radius, and preventive recommendations.

Codebase overview.

Understanding the repository structure and entrypoints.

Root Files
main.py

The Flask web server entrypoint. Serves the dashboard UI, handles GitLab webhooks, and initializes the APScheduler background cron job.

requirements.txt / Dockerfile

Dependencies (google-genai, flask, mcp) and containerization definitions for Cloud Run deployment.

Core Modules
agents/llm_helper.py

Centralized Gemini API wrapper. Manages all requests to gemini-2.5-pro with proper system prompting.

tools/gitlab_mcp_tools.py

MCP client implementation using the Python mcp SDK. Connects to the GitLab MCP Server via stdio transport to open Merge Requests.

static/app.js & style.css

Frontend dashboard logic. Handles polling, renders the mock terminal logs, and creates the typewriter animation effects.

Integration Guide

Deploy Always-On-CallSRE directly into your own infrastructure in minutes.

1

Clone & Install

git clone https://github.com/Saaalil/OnCallSRE.git
cd OnCallSRE
pip install -r requirements.txt
2

Configure Environment

Create a .env file in the root directory and add your Google AI Studio key.

GEMINI_API_KEY=your_google_genai_api_key
3

Connect GitLab MCP

To enable live MCP for creating real Merge Requests, update tools/gitlab_mcp_tools.py:

# Uncomment the real MCP client code and remove the mock responses.
# Requires: npx -y @modelcontextprotocol/server-gitlab
4

Deploy & Webhooks

Deploy the Flask app to Google Cloud Run or AWS ECS. Configure your GitLab project webhooks to point to:

https://your-deployed-url/webhook/gitlab