awscloudwatchreceiver

AWS CloudWatch Receiver

The AWS CloudWatch Receiver receives Cloudwatch events from AWS CloudWatch via the AWS SDK for Cloudwatch Logs

Status
Stability	alpha: logs, metrics
Distributions	contrib
Issues
Code coverage
Code Owners	@schmikei, @dyl10s | Seeking more code owners!
Emeritus	@djaglowski

Getting Started

This receiver uses the AWS SDK as mode of authentication, which includes Credentials File and IMDS authentication for EC2 instances.

Configuration

Top Level Parameters

Logs Parameters

Group Parameters

autodiscover and named are ways to control and filter which log groups and log streams which are collected from. They are mutually exclusive and are incompatible to be configured at the same time.

• autodiscover
  • limit: (optional; default = 50) Limits the number of discovered log groups. This does not limit how large each API call to discover the log groups will be.
  • prefix: (optional) A prefix for log groups to limit the number of log groups discovered.
    • Only one of prefix or pattern can be specified. If both are omitted, all log streams up to the limit are collected.
  • pattern: (optional) A case-sensitive substring (not a regular expression) that must be present in the log group names, used to limit the number of log groups discovered.
    • Only one of prefix or pattern can be specified. If both are omitted, all log streams up to the limit are collected.
  • account_identifiers: (optional) A list of AWS account IDs to filter log groups by. Only log groups from the specified accounts will be discovered.
  • include_linked_accounts: (optional; default = false) When using a monitoring account, set this to true to have autodiscovery return log groups in the accounts listed in account_identifiers. If this is true and account_identifiers contains a null value, all log groups in the monitoring account and in all linked source accounts are returned (behavior is handled by the AWS CloudWatch Logs API).
  • streams: (optional) If streams is omitted, then all streams will be attempted to retrieve events from.
    • names: A list of full log stream names to filter the discovered log groups to collect from.
    • prefixes: A list of prefixes to filter the discovered log groups to collect from.
• named
  • This is a map of log group name to stream filtering options
    • streams: (optional)
      • names: A list of full log stream names to filter the discovered log groups to collect from.
      • prefixes: A list of prefixes to filter the discovered log groups to collect from.

Metrics collection

Metrics are scraped on a configurable interval using the CloudWatch GetMetricData API. You can either list the metrics you want explicitly under queries, or let the receiver discover them automatically via discovery. The two options are mutually exclusive.

Scraper settings

Explicit queries (queries)

List every metric you want to collect. Each entry supports:

Auto-discovery (discovery)

Instead of listing metrics manually, the receiver can call ListMetrics to discover them automatically.

Statistics

The stats field controls which CloudWatch statistics are fetched and how they are represented in OpenTelemetry:

• Omitted (default): the four standard statistics — Sum, SampleCount, Minimum, Maximum — are fetched and combined into a single Summary data point per timestamp (sum, count, quantile 0.0 for minimum, quantile 1.0 for maximum). This matches the CloudWatch Metric Streams OpenTelemetry 1.0.0 format and costs 4 API sub-queries per metric per scrape.
• Explicit list: only the listed statistics are fetched, each costing one API sub-query. Each produces a separate Gauge data point on the same metric, identified by a stat attribute (e.g. stat = Average). Use this when you only need a subset of statistics to reduce AWS API costs. Different metrics in the same queries list can have different stats.

Standard statistics: Average, Sum, Minimum, Maximum, SampleCount. Extended statistics (percentiles, trimmed means, etc.): p99, p95, p50, tm99, wm99, etc. Extended statistics are only supported on metrics that explicitly enable them in CloudWatch — requesting an unsupported extended statistic returns no data.

Output format

All metrics follow the CloudWatch Metric Streams OpenTelemetry 1.0.0 format:

• Metric name: amazonaws.com/{Namespace}/{MetricName} (CloudWatch casing preserved)
• Resource attributes: cloud.provider = aws, cloud.region = <configured region>
• Data point attributes: Namespace, MetricName, and Dimensions (a nested key/value map, omitted when no dimensions are set)

Examples

Collect two EC2 metrics as Summary (all four standard statistics):

awscloudwatch:
  region: us-east-1
  metrics:
    collection_interval: 1m
    period: 60s
    queries:
      - namespace: AWS/EC2
        metric_name: CPUUtilization
        dimensions:
          InstanceId: i-1234567890abcdef0
      - namespace: AWS/EC2
        metric_name: NetworkIn

Collect specific statistics only (Gauge output, fewer API sub-queries per metric):

awscloudwatch:
  region: us-east-1
  metrics:
    collection_interval: 1m
    period: 60s
    queries:
      - namespace: AWS/EC2
        metric_name: CPUUtilization
        stats:
          - Average
          - p99
      - namespace: AWS/DynamoDB
        metric_name: SuccessfulRequestLatency
        dimensions:
          TableName: my-table
          Operation: GetItem
        stats:
          - p50
          - p95
          - p99

Auto-discover all EC2 metrics (Summary output):

awscloudwatch:
  region: us-east-1
  metrics:
    collection_interval: 5m
    period: 300s
    delay: 10m
    discovery:
      filters:
        namespace: AWS/EC2
      limit: 200

Logs Autodiscovery Example Configuration

awscloudwatch:
  region: us-west-1
  logs:
    poll_interval: 1m
    groups:
      autodiscover:
        limit: 100
        prefix: /aws/eks/
        streams:
          prefixes: [kube-api-controller]

Logs Autodiscovery with Account ID Filtering Example

awscloudwatch:
  region: us-west-1
  logs:
    poll_interval: 1m
    groups:
      autodiscover:
        limit: 100
        account_identifiers: ["123456789012", "987654321098"]
        include_linked_accounts: true
        prefix: /aws/lambda/

Logs Named Groups Example

awscloudwatch:
  region: us-west-1
  logs:
    poll_interval: 5m
    groups:
      named:
        /aws/eks/dev-0/cluster: 
          names: [kube-apiserver-ea9c831555adca1815ae04b87661klasdj]

Sample Configs

This receiver has a number of sample configs for reference.

1. Default

   • Minimal configuration of the receiver
   • Performs autodiscovery
   • Collects all log groups and log streams

2. Autodiscover Filtering Log Groups

   • Performs autodiscovery
   • Only collects log groups matching a prefix
   • Limits the number of discovered Log Groups

3. Autodiscover Filtering Log Streams

   • Performs autodiscovery for all Log Groups
   • Filters log streams

4. Named Groups

   • Specifies and only collects from the desired Log Groups
   • Does not attempt autodiscovery

5. Named Groups Filter Log Streams

   • Specifies the names of the log groups to collect
   • Does not attempt autodiscovery
   • Only collects from log streams matching a prefix