Status
Stability	deprecated
Deprecation of extension	Date: 2026-05-07
	Migration Note: Use the kafkareceiver with topic regex support instead. This component will be removed in a future release.
Distributions	contrib
Issues
Code coverage
Code Owners	@MovieStoreGuy

The kafkatopicsobserver extension is deprecated and will be removed in a future release. Use the kafkareceiver with topic regex support instead.

The Kafka topics observer extension is a Receiver Creator-compatible "watch observer" that will detect and report kafka topics in kafka cluster based on regex template. This observer watches available topics and matches them with the provided regex. If any change in available topics matching the regex is detected, the observer updates the endpoints list.

The following settings are required:

• topic_regex regex pattern of the topic name to subscribe to.

The following settings can be optionally configured:

• brokers (default = localhost:9092): The list of kafka brokers
• resolve_canonical_bootstrap_servers_only (default = false): Whether to resolve then reverse-lookup broker IPs during startup
• protocol_version (default = 2.1.0): Kafka protocol version e.g. 2.0.0
• client_id (default = "otel-collector"): The client ID to configure the Kafka client with.
• topics_sync_interval (default 5s)
• tls: see TLS Configuration Settings for the full set of available options.
• auth
  • plain_text (Deprecated in v0.123.0: use sasl with mechanism set to PLAIN instead.)
    • username: The username to use.
    • password: The password to use
  • sasl
    • username: The username to use.
    • password: The password to use
    • mechanism: The sasl mechanism to use (SCRAM-SHA-256, SCRAM-SHA-512, AWS_MSK_IAM_OAUTHBEARER, or PLAIN)
    • aws_msk
      • region: AWS Region in case of AWS_MSK_IAM_OAUTHBEARER mechanism
  • tls (Deprecated in v0.124.0: configure tls at the top level): this is an alias for tls at the top level.
  • kerberos
    • service_name: Kerberos service name
    • realm: Kerberos realm
    • use_keytab: Use of keytab instead of password, if this is true, keytab file will be used instead of password
    • username: The Kerberos username used for authenticate with KDC
    • password: The Kerberos password used for authenticate with KDC
    • config_file: Path to Kerberos configuration. i.e /etc/krb5.conf
    • keytab_file: Path to keytab file. i.e /etc/security/kafka.keytab
    • disable_fast_negotiation: Disable PA-FX-FAST negotiation (Pre-Authentication Framework - Fast). Some common Kerberos implementations do not support PA-FX-FAST negotiation. This is set to false by default.
• metadata
  • refresh_interval (default = 10m): How often to refresh Kafka topic/partition metadata