Here is how we tackled the problem by introducing a dynamic transformation layer into our Amazon Connect and Kinesis pipeline.

The Existing Architecture

Our original pipeline was built to stream contact center metrics directly into our data lake for analytics.

How the Data Flow Worked

1. Amazon Connect: When a customer interacts with an agent, Amazon Connect generates a Contact Trace Record (CTR) detailing the interaction.

2. Kinesis Data Streams: The CTR data is pushed to Kinesis Data Streams, protected using AWS KMS encryption.

3. Kinesis Data Firehose: Firehose consumes the stream records to batch and prepare them for delivery.

4. AWS Glue: Firehose uses AWS Glue for schema-related processing to understand the data structure.

5. Apache Parquet Conversion: Firehose converts the processed records into columnar Apache Parquet format.

6. Amazon S3: The resulting Parquet files are written to an Amazon S3 bucket.

7. Databricks: Databricks reads the Parquet files from S3 for downstream analytics and processing.

The Problem: Fixed Schema in AWS Glue

The pipeline itself was working, but we ran into a structural challenge as our contact flows evolved.

Our AWS Glue schema definition was fixed. When the incoming CTR payload structure changed—such as introducing a new attribute or modifying nested fields—the rigid schema could no longer keep up cleanly.

For instance, consider how a simple JSON payload change impacts a strict schema setup:

// Existing JSON Structure
{
  "fieldA": "valueA",
  "fieldB": "valueB"
}

// Evolved JSON Structure
{
  "fieldA": "valueA",
  "fieldB": "valueB",
  "fieldC": "valueC"
}

The issue isn't just that JSON schemas change; it is that a fixed Glue schema can quickly turn into a maintenance bottleneck. If incoming records contain new elements or altered structures, downstream formatting can fail or drop fields unless the schema definitions are manually or constantly updated. We needed a way to handle schema evolution more gracefully without constantly breaking our delivery pipeline.

Our Proposed Architecture

To solve this, we proposed introducing an AWS Lambda function between Kinesis Data Firehose and AWS Glue. This acts as a transformation and detection layer, allowing us to inspect and normalize incoming data structures dynamically before Glue processes them.

Step-by-Step Through the New Flow

1. Amazon Connect: Generates the CTR for customer-agent interactions.

2. Kinesis Data Streams: Ingests the CTR data, secured with KMS encryption.

3. Kinesis Data Firehose: Receives the stream records and prepares them for delivery.

4. Lambda Transformation Layer: Instead of sending raw, unpredictable payloads straight to Glue, Firehose passes the records through an AWS Lambda function. Lambda examines the incoming record, identifies the relevant object ID and JSON structure, and prepares the payload so downstream services can handle it properly.

5. AWS Glue: Receives the dynamically prepared information to provide the correct schema mapping.

6. Firehose Data Conversion: Armed with the proper schema handling, Firehose converts the data into Apache Parquet format.

7. Amazon S3: Stores the finalized Parquet files.

8. Databricks: Consumes the Parquet data from S3 for downstream analytics.

Original vs. Proposed Architecture

• Original: Connect → CTR → Kinesis Data Streams → Firehose → Glue → Parquet → S3 → Databricks

  • Limitation: Relies entirely on a fixed, hard-coded Glue schema.

• Proposed: Connect → CTR → Kinesis Data Streams → Firehose → Lambda → Glue → Firehose → Parquet → S3 → Databricks

  • Improvement: Introduces a flexible Lambda transformation layer to handle schema variations before Glue processing.

Benefits and Trade-offs

Introducing Lambda into the middle of a Firehose delivery stream gave us much better control over data payloads, but it came with trade-offs worth considering:

• Pros:

  • Greater flexibility when handling evolving JSON structures.

  • Reduced maintenance overhead for rigid, static schemas.

  • Better visibility into record payloads before storage conversion.

• Cons & Considerations:

  • Additional architectural complexity with a compute layer inside the streaming pipeline.

  • Lambda execution latency and concurrency limits to monitor.

  • Added monitoring and error-handling requirements for transformation failures.

Key Takeaways

Rigid schemas are often the silent bottleneck of real-time data pipelines. By introducing an AWS Lambda transformation layer between Kinesis Data Firehose and AWS Glue, we decoupled our ingestion pipeline from strict schema limitations, making our contact center data lake much more resilient to change.

Recently, our application teams required an update to our Amazon Connect Outbound Campaigns architecture. What started as a straightforward event pipe needed to evolve into a multi-stage, fan-out event-processing pipeline for contact ID and attribute enrichment.

In this post, I will walk through how we approached this architectural shift, mapped out the components, implemented them using our tiered Terraform repository structure, and managed the complexities of multi-account deployments.

Table of Contents

1. Introduction

2. Our Terraform Repository Architecture

3. The Existing Amazon Connect Outbound Architecture

4. Why the Architecture Needed to Change

5. The New Architecture

6. Why SNS + SQS Was Introduced

7. Terraform Implementation

8. Multi-Account Deployment

9. Infrastructure as Code vs. Console-Based Infrastructure

10. Engineering Considerations

11. Lessons Learned as a Platform Engineer

12. Key Takeaways

1. Introduction

When product requirements shift, infrastructure must adapt without introducing drift or deployment friction. For a platform engineer, handling a request like "we need to enrich contact attributes before they hit downstream systems" means looking beyond the immediate AWS console clicks. We have to design an infrastructure building block that is clean, secure, least-privileged, and easily stampable across staging, production, and multiple customer-facing AWS accounts.

2. Our Terraform Repository Architecture

To maintain consistency and avoid copy-pasting Terraform code across dozens of repositories, our team enforces a structured, tiered repository hierarchy:

• Terraform Public Modules: Base, upstream modules defining core building blocks.

• JFrog Organization Repository: Internally versioned and vetted modules stored in our organization's JFrog registry.

• Wrapper Repository: Environment- or pattern-opinionated wrappers that bind internal standards to modules.

• Service-Specific Repository: The actual implementation repo (e.g., Connect, Networking, Routing repositories) where application-specific infrastructure is composed using our wrappers and modules.

3. The Existing Amazon Connect Outbound Architecture

Previously, our Amazon Connect outbound campaign flow was intentionally simple:

An Amazon EventBridge Pipe directly connected outbound campaign events from Amazon Connect to an Amazon Kinesis Data Stream. This worked well for basic streaming requirements where raw events required zero intermediate transformation or validation.

4. Why the Architecture Needed to Change

New requirements introduced a need for Contact ID and Contact Attribute Processing.

The direct flow from EventBridge to Kinesis was no longer sufficient because upstream contact data needed to be intercepted, parsed, enriched, and fanned out to distinct consumers (such as user profile updaters and secondary logging or processing paths) before reaching the final data stream.

5. The New Architecture

To support enrichment and parallel consumer workflows, we expanded the pipeline into an event-driven fan-out pattern.

Service Responsibilities:

• Amazon Connect: Generates outbound contact campaign events.

• Amazon EventBridge: Receives outbound events and routes them to the initial processing handler.

• Enrichment Lambda: Processes incoming contact IDs and contact attributes.

• Amazon SNS: Acts as a fan-out publishing hub for the enriched messages.

• SQS #1 & SQS #2: Decouple the SNS publisher from downstream consumers, offering buffering and queue isolation.

• Update Profile Lambda: Consumes messages from SQS #1 to update contact profiles before pushing records into Kinesis.

• Update Lambda: Consumes messages from SQS #2 to perform additional contact-related processing.

• Kinesis Data Streams: Receives the final processed events for downstream ingestion.

6. Why SNS + SQS Was Introduced

Introducing Amazon SNS combined with multiple Amazon SQS queues provides key architectural advantages:

• Fan-out Capability: A single enriched event from the Lambda function can be broadcast to multiple independent subscribers simultaneously via SNS.

• Decoupling & Asynchronous Processing: Producers do not block or wait for consumers to finish processing, shielding upstream systems from downstream latency or transient outages.

• Failure Isolation & Retries: If a consumer Lambda fails while processing a batch from an SQS queue, messages are retained safely in the queue according to visibility timeout configurations, preventing data loss.

7. Terraform Implementation

Translating this architecture into our service-specific repository means orchestrating multiple AWS resources cleanly. Rather than writing raw, repetitive resource definitions, we leverage our internal wrappers.

An illustrative example of how we compose parts of this pipeline in Terraform:

module "enrichment_lambda" {
  source = "app.jfrog.io/our-org/lambda-wrapper/aws"
  version = "1.2.0"

  function_name = "connect-enrichment-${var.environment}"
  handler       = "index.handler"
  runtime       = "nodejs18.x"
  role_arn      = aws_iam_role.lambda_execution.arn
  
  environment_variables = {
    SNS_TOPIC_ARN = aws_sns_topic.enrichment.arn
  }
}

resource "aws_sns_topic" "enrichment" {
  name = "connect-enriched-events-${var.environment}"
}

resource "aws_sqs_queue" "profile_update" {
  name = "connect-profile-update-${var.environment}"
}

resource "aws_sns_topic_subscription" "sqs_subscription" {
  topic_arn = aws_sns_topic.enrichment.arn
  protocol  = "sqs"
  endpoint  = aws_sqs_queue.profile_update.arn
}

Other components managed within the repository include EventBridge rule targets, SQS-to-Lambda event source mappings, least-privilege IAM roles, and Kinesis stream configurations.

8. Multi-Account Deployment

Because this infrastructure pattern runs across multiple AWS accounts and environments (such as staging and production), hardcoding names, ARNs, or account IDs is prohibited.

• Environment-Specific Variables: Inputs like environment tags, naming prefixes, and scaling thresholds are passed down via Terraform variables (var.environment).

• Dynamic ARNs: IAM policies and SNS subscription endpoints dynamically reference resource attributes (aws_sns_topic.enrichment.arn) rather than relying on static strings.

• Provider Configuration: Multi-account provider aliasing ensures our wrapper modules stamp identical architecture reliably into whichever target AWS account our pipeline authenticates against.

9. Infrastructure as Code vs. Console-Based Infrastructure

Managing an event-driven topology of this size manually via the AWS Console introduces significant operational risks:

10. Engineering Considerations

When deploying this architecture, our Platform Engineering team pays close attention to several production criteria:

• IAM & Least-Privilege: We strictly scope permissions—ensuring EventBridge can only invoke the Enrichment Lambda, Lambda can only publish to the specific SNS topic, and SQS queues only permit messages from authorized SNS topics or trigger their designated consumer Lambdas.

• Reliability: A consideration for production: Configure appropriate SQS visibility timeouts that exceed the maximum execution timeout of the downstream consumer Lambdas, and set up Dead-Letter Queues (DLQs) to catch poisoned messages.

• Observability: CloudWatch metrics on SQS queue depths (e.g., ApproximateNumberOfMessagesVisible) and Lambda error rates are critical for spotting processing bottlenecks across the pipeline.

• Scalability: The asynchronous handoff between SNS and SQS ensures spikes in Amazon Connect outbound call volume are safely buffered without overwhelming consumer functions.

11. Lessons Learned as a Platform Engineer

• Requirements evolve: Event-driven architectures often start simple (like a direct pipe) and grow complex as business logic requires intermediate data enrichment. Building modularly from the start makes these pivots easier.

• Abstraction must balance reusability: When wrapping modules for application teams, hide unnecessary complexity while exposing critical tuning parameters (like timeout and memory sizes for Lambda).

• Consistency matters: Multi-account deployments amplify small configuration mistakes; enforcing infrastructure changes through code review protects every environment equally.

12. Key Takeaways

• Decoupling scales: Inserting SNS and SQS between producers and consumers protects your pipelines from downstream failures and traffic spikes.

• Platform engineering creates leverage: Reusable Terraform wrappers prevent teams from reinventing event-driven patterns for every new account.

• Keep code DRY: Organizing Terraform configurations into a structured repository hierarchy makes maintaining multi-account AWS architectures sustainable.

What if you could automate the entire triage and fix, keep a strict audit trail, completely ban SSH, and still keep a human in the loop for safety—all via Microsoft Teams?

Let’s look at a production-grade, multi-region architecture designed to do exactly that.

The Core Design Principles

Before looking at the workflow, this architecture is anchored on a few non-negotiable security and operational pillars:

• No SSH Allowed: Absolutely zero direct SSH access to the workloads. All actions happen securely via AWS Systems Manager (SSM).

• Human-Approved Actions: Automation doesn't mean blindly running scripts. Critical actions require an authorized engineer to click a button.

• Least Privilege & Cross-Account Isolation: The system isolates the remediation logic inside a central Automation Control Plane account, assuming minimal IAM roles to interact with target workload accounts.

• Audit Everything: Every single alert, payload validation, human click, and command output is logged immutably.

Architecture Blueprint

Step-by-Step Workflow Breakdown

The entire lifecycle—from detecting a fault to fixing it—is broken down into distinct, decoupled phases managed seamlessly by AWS Step Functions.

1. Alert Detection & Secure Ingress

• Detection: Prometheus and Node Exporters continuously gather metrics across the EC2 fleet. When an anomaly or threshold is met, Grafana evaluates the alert rule.

• Ingress: Grafana sends a secure HTTPS Webhook payload. This hits CloudFront (global entry) and passes through AWS WAF (handling rate limiting and IP allowlisting) before triggering an API Gateway HTTP API.

2. Validation & Parsing

• Handler Lambda: Validates the payload’s timestamp, checks a cryptographic nonce to prevent replay attacks, verifies the HMAC signature, and generates a unified correlation ID.

• Alert Parser Lambda: If valid, the Step Function workflow starts. The alert parser extracts critical metadata: instance_id, private_ip, region, account_id, and metric details.

• Instance Resolver Lambda: Maps the alert data to the exact EC2 target, calling DescribeInstances if it only has a private IP to grab instance tags, state, and OS details.

3. Guardrails & The Human Element

• Security & Guardrail Validation: Before moving an inch forward, a dedicated safety check runs. Is this a protected instance? Is the requested action allowed? Is the SSM Agent online?

• Orchestration (Step Functions): AWS Step Functions acts as the brain, managing the logic gates and triggering a pause to wait for human interaction.

• Human Approval via Teams: A Teams Bot sends a highly descriptive Adaptive Card to an authorized Microsoft Teams channel. The card displays live alert details alongside clear interactive choices: Approve, Reject, Ignore, Restart Service, or Kill Process. If a timeout occurs, it defaults to a safe Auto-Reject status.

4. Cross-Account Execution

• Executor Lambda: Once an operator clicks "Approve", the workflow resumes. The Executor Lambda assumes a cross-account IAM role into the specific workload account target.

• AWS Systems Manager (SSM): Rather than manual execution, it leverages approved, pre-defined SSM Documents to run the remediation script (Run Command) with strict concurrency control.

5. Collection, Notification & Audit

• Reporter Lambda: Collects the execution logs and command results from SSM, parsing whether the remediation succeeded or failed.

• Teams Notification: Sends a final success/failure summary back to the Microsoft Teams channel, noting execution time and next steps.

• Auditor Lambda: Writes the complete history to an immutable S3 bucket (with Object Lock), updates a central DynamoDB table, and closes out the incident.

The Centralized Logging & Monitoring Layer

To ensure compliance and continuous debugging, every single phase pipes data into a centralized observability stack:

• CloudWatch Logs & Alarms: Centralizes logs from all Lambda functions and monitors for execution errors or timeouts.

• CloudTrail: Audits every single API call and cross-account STS role assumption.

• Amazon Athena & QuickSight: Allows the operations team to run analytical SQL queries over historical audit data and view remediation success metrics via interactive dashboards.

• AWS X-Ray: Provides end-to-end distributed tracing across the API Gateway, Step Functions, and various Lambda microservices.

Conclusion

By shifting away from traditional SSH runbooks and moving towards an event-driven, microservice-based remediation model, you get the best of both worlds: rapid incident response times and airtight enterprise security. Your engineering team stays in full control right from their chat client, and your infrastructure remains a hardened fortress.

What are your thoughts on using interactive chat cards for infrastructure remediation? Let me know in the comments below!

By the time the standard AWS monthly budget alert triggers, the damage is already in the thousands of dollars.

While AWS provides native solutions like AWS Cost Anomaly Detection, these tools often act as a black box. They use complex, proprietary machine learning models that can take days to train, sometimes missing sudden, sharp spikes in lower-spending accounts or generating a flood of alert fatigue for minor deviations.

I wanted something different for our production environment: a lightweight, serverless, deterministic, and AI-supplemented billing monitor built entirely using native AWS primitives and our existing observability stack.

The Engineering Guardrails
When mapping out this Proof of Concept (POC), I enforced strict constraints to keep our operational footprint near zero:
1. Fully Serverless & Stateless: No databases, no persistent queues, and no long-running container clusters.
2. Zero Added Infrastructure Drag: Absolutely NO API Gateway, DynamoDB, S3 buckets, RDS databases, AWS Step Functions, or OpenSearch clusters. The system must run entirely within ephemeral execution contexts.
3. Deterministic Logic, AI Explanation: AI should never decide if an anomaly happened—that is a math problem. Instead, AI should only be invoked to explain the anomaly after the math flags it. This keeps inference costs trivial.
4. Native Observability Integration: It must feed directly into our existing self-hosted Grafana instance and emit standard CloudWatch metrics.
5. Actionable ChatOps: Notifications must land directly in Microsoft Teams via Modern Workflows using rich Adaptive Cards, bypassing old SNS-to-email pipelines.

Here is the engineering story of how I designed, implemented, debugged, and rolled out this intelligent cloud financial operations (FinOps) guardrail.

The System Architecture
A reliable production monitoring tool must be simple. By stripping away extraneous structural layers, we eliminated cold starts, complex state machines, and data synchronization issues.
The entire data and control flow executes within a single daily cron cycle:

Deep-Dive: Service Responsibilities

1. Amazon EventBridge Scheduler

Acts as my reliable, serverless heartbeat. It triggers my central Lambda function daily at a designated time (e.g., 06:00 UTC), ensuring I calculate costs as soon as the previous day’s bill stabilizes.

2. AWS Lambda (Core Engine)

The single stateless computing core written in clean, modular Python. It manages the orchestration: retrieving data from Cost Explorer, calculating anomalies, sending telemetry to CloudWatch, invoking Bedrock if necessary, and assembling the payload for Microsoft Teams.

3. AWS Cost Explorer API

The source of truth for all cost and usage anomalies. I query it dynamically to extract the last 7 days of raw spending data, broken down by individual AWS services.

4. CloudWatch Metrics & Logs

Instead of standing up a dedicated database to track spending history, I offload persistence entirely to CloudWatch Custom Metrics. Historical metrics provide the long-term trend data, while structured CloudWatch Logs store the raw JSON strings for auditability.

5. Amazon Bedrock (Anthropic Claude 3 Haiku)

My context-aware on-call assistant. When an anomaly is confirmed by my deterministic math formulas, Bedrock evaluates the service breakdown and writes a natural-language brief explaining exactly which service is driving the spike.

6. Microsoft Teams Workflow

My alerting destination. I use the updated Office 365 Workflows engine to accept incoming JSON webhooks and render highly structured, readable, color-coded Adaptive Cards directly inside my engineering channels.

7. Existing Self-Hosted Grafana

The analytical lens. Rather than adding a new dashboarding engine, I use the native Amazon CloudWatch plugin inside my pre-existing Grafana server to construct beautiful executive overviews and technical trend panels.

The Business Logic: Dual-Condition Anomaly Detection

A naive billing alert simply triggers when today's cost is higher than yesterday's. This is an operational anti-pattern that creates constant false alarms.

For instance, if your cost jumps from $5 to $10 due to a routine automated backup cycle, that is a 100% increase, but it is completely insignificant in terms of real budget impact. Conversely, if your infrastructure steadily scales up over a week, comparing today only against yesterday might mask a massive cumulative spike.

To build a reliable indicator, my system requires both of the following conditions to evaluate to True before raising an alarm:

Condition 1: Day-over-Day (DoD) Percentage Variance

This measures sudden, immediate cost explosions between today and yesterday.

Condition 2: 5-Day Rolling Average Variance

This ensures that today's spike is a significant deviation from the baseline established over the previous work week.

The Decision Matrix (Alert = Condition 1 AND Condition 2)

Production Guardrail: Divide-by-Zero Protection

In newly provisioned cloud environments, sandbox accounts, or low-traffic regions, yesterday's cost or the 5-day baseline average might be exactly $0.00. Mathematically, dividing by zero causes unhandled execution exceptions (ZeroDivisionError), crashing the execution flow.

Our core detection logic implements an explicit protection policy:

def calculate_percentage_variance(current_val: float, baseline_val: float) -> float:
    if baseline_val <= 0:
        return 100.0 if current_val > 0 else 0.0
    return ((current_val - baseline_val) / baseline_val) * 100.0

• If the baseline is $0.00 and current spending jumps to any positive value (e.g., $15.00), it represents a new cost center, triggering a 100% variance jump.

• If both baseline and current spending are $0.00, the variance is tracked as 0%.

Project Directory & Module Architecture

To ensure this codebase remains clean, testable, and maintainable, we decoupled every logical boundary into dedicated files.

billing-monitor/
├── lambda_function.py        # Central Lambda Orchestrator & Handler
├── config.py                 # Environment Variables & Hardcoded Thresholds
├── logger.py                 # Structured Global CloudWatch JSON Logger
├── cost_explorer.py          # Wrapper for AWS Cost Explorer Boto3 API
├── anomaly_detector.py       # Pure Deterministic Math Engine
├── cloudwatch_metrics.py     # Custom Metrics Dispatcher
├── bedrock_service.py        # GenAI Prompt & Context Processor
├── teams_notifier.py         # HTTP Webhook Handler for Teams Adaptive Cards
├── tests/
│   └── test_detector.py      # Local Validation Matrix & Mock Testing Unit
└── grafana/
    └── dashboard.json        # Production-ready Dashboard Definition Export

Module Breakdown

• lambda_function.py: The system gateway. It boots up, triggers the data collection lifecycle, balances states, catches top-level errors, and cleans up execution dependencies.

• config.py: Single source of truth for runtime constants (e.g., anomaly thresholds, Bedrock model definitions, Teams webhook URLs).

• logger.py: Standardizes formatting so that all outputs are generated as searchable JSON blocks in CloudWatch Logs.

• cost_explorer.py: Abstracts complex boto3 parameters into a clean, direct interface that returns pure dictionary maps.

• anomaly_detector.py: A pure, dependency-free module containing our mathematical variance algorithms and decision matrices.

• cloudwatch_metrics.py: Handles high-velocity metric construction, packing multiple dimensions for optimal data ingestion.

• bedrock_service.py: Contains the prompt engineering blocks and structures payload variables for safe interactions with LLM endpoints.

• teams_notifier.py: Prepares the structural JSON styling requirements of Microsoft Teams Workflows.

Production Alerting and Alarms
Relying entirely on a notification script means you risk missing errors if the system itself breaks down. To ensure full coverage, configure the following native CloudWatch Alarms alongside your dashboard metrics:

1. Hard Anomaly Alarm Metric: AnomalyStatus

• Metric: AnomalyStatus

• Threshold: > 0 for 1 consecutive period.

• Action: If our mathematical calculation detects an anomaly, this alarm changes state immediately. It serves as a critical operational fallback, with notifications routed to an SNS queue or PagerDuty.

2. Monitoring Pipeline Error Fail-Safe Metric:

• Metric: Errors under namespace AWS/Lambda for dimension FunctionName: BillingMonitor.

• Threshold: >= 1 for 1 evaluation loop.

• Action: Triggers an alert if our custom python script throws an unhandled exception, ensuring visibility if API changes or runtime crashes break our monitoring system.

Production Edge Cases and Challenges

Building a production-ready system means anticipating where things will break in the real world. During our testing phases, we ran into several interesting edge cases that forced us to refine our design.

Challenge 1: Bedrock Claude 3 JSON Formatting Deviations

The Issue: LLMs can be unpredictable. Despite specifying strict output constraints, Claude occasionally wrapped its responses in unwanted markdown blocks.

2. Multi-Account Aggregation Strategies

In complex organizational settings, deploying separate monitoring functions across dozens of distinct child accounts quickly becomes a management nightmare.

Instead, leverage AWS Organizations to deploy this system a single time inside your centralized Billing or Management Master Account. From there, modify your Cost Explorer ingestion queries to group data fields by the LINKED_ACCOUNT dimension. This allows you to track and segment anomalies across your entire organization from a single, centralized point of control.

Conclusion and Next Steps

By designing with strict constraints, we built an intelligent, serverless cloud financial monitor that keeps operational overhead near zero.

By avoiding complex state databases, API gateways, and web application servers, we eliminated common security vectors and maintenance headaches.

Our dual-condition verification logic acts as a dependable mathematical filter, ensuring we only invoke Bedrock's generative text models when a true anomaly is confirmed. This keeps our monthly AI inference cost to pennies, while delivering clear, context-aware engineering summaries directly to our team via Microsoft Teams.

The system is fully auditable, emits native CloudWatch metrics, and plugs seamlessly into our existing Grafana setup.

What's Next?

If you're looking to extend this Proof of Concept, consider adding these enhancements:

• Automated Remediation Guardrails: Modify the Lambda function to trigger automated incident response runbooks when severe anomalies occur—such as automatically isolating or spinning down unmanaged development resources.

• Tag-Based FinOps Allocation: Update your Cost Explorer queries to cross-reference your organization's custom billing tags (e.g., Owner, Environment, Project). This would allow the AI engine to call out the specific team or project responsible for a spending spike.

Have you experienced an unexpected cloud billing spike that went unnoticed for too long? How is your team currently tackling cloud cost tracking and observability? Let's discuss in the comments below!