Skip to main content

Confluent integration

Connect your existing Confluent Cloud Apache Kafka instance to Squid

To access your Confluent Cloud through Squid, first add the integration to Squid in the console.

  • In the Squid Console, navigate to the Integrations page and select the Confluent Kafka integration.

  • Provide the following details:

    • Integration ID - Choose an ID that is brief and helps identify the integration.
    • Bootstrap servers - A comma-separated list of servers and ports. You can find this value in your cluster settings.
    • Key - Your Kafka API Key. Click on your cluster and then click your API Keys to find existing keys or create a new one.
    • Secret - Your Kafka API secret. This is securely stored in Squid Secrets.
    • Avro schema registry - In the case that you would like to consume or produce Avro messages, provide the URL of your Avro schema registry. You can find this value in your cluster settings.
  • Click Test connection to verify the integration information. Once verified, click Add integration.

Confluent integration

To access the queue from the client, create a reference to a QueueManager using queue and passing the topic name and integration ID:

Client code
const queue = squid.queue('topic-name', 'confluent-integration-id');

To read messages from a queue, use the consume method. This returns an Observable that updates whenever a new message is posted to the queue.

Client code
const topicMessagesObs = queue.consume<MyType>();

topicMessagesObs.subscribe((message: MyType) => {
console.log(message);
});
Note

When you subscribe to a topic, the Observable you receive is configured to only include messages produced after the subscription is established. This behavior is due to the subscription process involving a server call, which introduces an approximate delay of ~100ms. Therefore, you can expect a short wait before the Observable starts delivering new messages.

To prevent memory leaks, complete your Observable when no longer in use:

Client code
topicMessagesObs.unsubscribe();

To add messages to a queue, use the produce method. This method takes an array of messages that are then consumed by clients subscribed to the queue.

Client code
queue.produce(['hello', 'world']);

Securing your Confluent topics

Accessing your Confluent topics on the client requires security functions.

To secure a Squid queue topic, use the @secureTopic decorator, passing the topic name and the type of action. The following code allows read and write access to a queue for the 'topic-name' topic:

Backend code
import { SquidService, secureTopic } from '@squidcloud/backend';

export class ExampleService extends SquidService {
@secureTopic('topic-name', 'all', 'confluent-integration-id')
allowTopicAccess(): boolean {
return true;
}
}

To learn more about security functions, see the Queue security docs.