Setup and Configuration.

Kafka is an open source streaming platform that you have to host it and configure it yourself. Makes it a little bit complex to manage and configure but gives you more control and better performance.

Kinesis is a a fully-managed streaming processing service that’s available on AWS. The only options you need to configure is number of shards and the number of days you need to keep the data.

Cost:

Apache Kafka requires that you do all the setup and configuration yourself. There will learning curve at the beginning and it will take some time to tune the platform for your use case.

Kinesis is a cloud service. Thus, the cost for setup and configuration is fairly less but it won’t be very efficient for certain use cases on the long run.

Performance:

Kafka’s performance is better given the same price. The reason behind this is that Kinesis needs to write each message synchronously to 3 different machines (availability zones) and this is costly in terms of latency and throughput.

Maximum Retain Period:

Kafka can store records forever (In Fact, there is nothing called “forever” on planet earth. but you got what I mean. right?).

On the other hand, An Kinesis stream stores records from 24 hours by default, up to 168 hours (7 days only).

Powered By:

Kafka: linkedin, pinterest, yahoo, spotify, netflex, tumblr, paypal …. and more

Kinesis: Nordstrom, dash, supercell, NDN … and more