AWS Kinesis Stream
This module allows CrowdSec to acquire logs from a Kinesis stream.
Configuration example
To monitor a stream:
source: kinesis
stream_name: my-stream
labels:
type: mytype
To monitor a stream using the enhanced fan-out API:
source: kinesis
stream_arn: "arn:aws:kinesis:region:000000000000:stream/my-stream"
use_enhanced_fanout: true
consumer_name: crowdsec-agent
labels:
type: mytype
If your stream is written to by a Cloudwatch subscription filter, you will need to pass the from_subscription parameter, or crowdsec won't be able to parse the content of the message.
Look at the configuration parameters to view all supported options.
Parameters
stream_name
The name of the kinesis stream you want to read logs from.
Required when use_enhanced_fanout is false.
stream_arn
The ARN of the kinesis stream you want to read logs from.
Required when use_enhanced_fanout is true
use_enhanced_fanout
Whether to use enhanced fan-out (dedicated throughput for a consumer) or not.
This option will incur additional AWS costs.
Defaults to false
consumer_name
Name of the consumer.
Required when enhanced_fan_out is true.
from_subscription
Whether the logs are coming from a Cloudwatch subscription filter or not.
When Cloudwatch writes logs to a kinesis stream, they are base64-encoded and gzipped, and the actual log message is part of a JSON object.
Defaults to false.
aws_profile
The AWS profile to use, relies on your ~/.aws/config/.
Optional, the data source will automatically the standard AWS env vars if present.
aws_config_dir
The path to your ~/.aws/, defaults to /root/.aws.
Optional, the data source will automatically the standard AWS env vars if present.
aws_region
The AWS region.
Optional, the data source will automatically the standard AWS env vars if present.
source
Must be kinesis
DSN and command-line
This datasource does not support acquisition from the command line.
