Azure EventHubs

This page describes connection Lenses to Azure EventHubs.

Azure EventHubs only support delete or compact as a topic cleanup policy.

1

Create a Data Integration API key

• Add a shared access policy

  • Navigate to your Event Hub resource and select Shared access policies in the Settings section.

  • Select + Add shared access policy, give a name, and check all boxes for the permissions (Manage, Send, Listen)

• Once the policy is created, obtain the Primary Connection String, by clicking the policy and copying the connection string. The connection string will be used as a JAAS password to connect to Kafka.

• The bootstrap broker [YOUR_EVENT_HUBS_NAMESPACE].servicebus.windows.net:9093

Configure Provisioning

2

Set the following in the provisioning.yaml

Due to Azure EventHubs limitation, Pricing tier for EventHub has to be at least Standard.

Environment variables are supported; escape the dollar sign

sslKeystorePassword:
  value: "\${ENV_VAR_NAME}"

With environment variable

First, set the environment variable

Note that "\" at "$ConnectionString" is set additionally to escape the $ sign.

terminal

export SASL_JAAS_CONFIG=org.apache.kafka.common.security.plain.PlainLoginModule required username="\$ConnectionString" password="Endpoint=sb://[SB_URL]/;SharedAccessKeyName=[KEY_NAME];SharedAccessKey=[ACCESS_KEY]";

provision.yaml

kafka:
- name: kafka
  version: 1
  tags: [my-tag]
  configuration:
    kafkaBootstrapServers:
      value:
        - SASL_SSL://[YOUR_BOOTSTRAP_SERVER]
        - SASL_SSL://[YOUR_BOOTSTRAP_SERVER]
    saslJaasConfig:
      value: '${SASL_JAAS_CONFIG}'
    saslMechanism:
      value: PLAIN
    protocol:
      value: SASL_SSL

Without environment variable

provision.yaml

connections:
  kafka:
  - name: Kafka
    version: 1
    tags: [my-tag]
    configuration:
      kafkaBootstrapServers:
        value:
          - SASL_SSL://[YOUR_BOOTSTRAP_SERVER]
          - SASL_SSL://[YOUR_BOOTSTRAP_SERVER]
      saslJaasConfig:
        value: org.apache.kafka.common.security.plain.PlainLoginModule required username="\$ConnectionString" password="Endpoint=sb://[SB_URL]/;SharedAccessKeyName=[KEY_NAME];SharedAccessKey=[ACCESS_KEY]";
      saslMechanism:
        value: PLAIN
      protocol:
        value: SASL_SSL