Relative Content

Tag Archive for apache-flinkflink-streamingamazon-kinesisamazon-kinesis-analytics

Preserving order of records on KinesisStreamsSink sink for Apache Flink in the face of write errors

I am writting a Flink application where order of the output records is paramount. Inside a certain batch of records it’s OK if they arrive out of order but batch 2 should never arrive before batch 1. I need to output them to an AWS Kinesis Stream and, for each key (which should always be written to the same shard) I cannot afford to lose the order of the records. I am using, for this, the KinesisStreamsSink and the way I am creating one looks like this :