Kafka Topics, Partitions and Offsets Explained

Kafka Topics, Partitions and Offsets Explained

Stephane Maarek

4 года назад

149,565 Просмотров

Learn about Apache Kafka Topics, Partitions and Offsets and their meanings!
If you want to learn more: https://links.datacumulus.com/apache-kafka-coupon
Get the Learn Apache Kafka for Beginners v3 course at a special price!

Don’t forget to subscribe to get more content about Apache Kafka and AWS!

I'm Stephane Maarek, a consultant and software developer, and I have a particular interest in everything related to Big Data, Cloud and API. I sat on the 2019 Program Committee organizing the Kafka Summit. I'm also an AWS Certified Solutions Architect, Developer, SysOps Administrator, and DevOps Engineer.

My other courses are available here: https://courses.datacumulus.com/

Follow me on social media:
LinkedIn - https://www.linkedin.com/in/stephanemaarek/
Twitter - https://twitter.com/stephanemaarek/
Medium - https://medium.com/@stephane.maarek

Тэги:

#Udemy #Stephane_Maarek #apache_Kafka #Kafka #learn #topics #partitions #offsets #tutorial #explained #introduction #stream #data
Ссылки и html тэги не поддерживаются


Комментарии:

roli agrawal
roli agrawal - 02.10.2023 15:26

So as u said once it’s written at offset we can not update it that means the same data we can not update it?

Ответить
Padmanathan Ramasamy
Padmanathan Ramasamy - 19.02.2023 06:04

In your truck_gps example, you have created 10 partition and created truckId as a key. In this case the first 10 unique truck id will stored on each partition. what happen when we get the 11th truckId to which partition it will go?

Ответить
Abhigyan Gaming
Abhigyan Gaming - 15.01.2023 13:37

Thanks a lot for these excellent lectures ❣

Ответить
Xiaowei Lu
Xiaowei Lu - 14.11.2022 21:36

Hey I was watching a Kafka course from Oreilly and found the truck_gps example familiar. Then it turned out to be exactly your course! Very good explanation! thank you!

Ответить
Leo Yang
Leo Yang - 25.04.2022 18:35

your videos are the best in explaining how kakfa works, clear and neat.

Ответить
Sushma Kl
Sushma Kl - 12.04.2022 12:25

how do I collect data from Kafka topic as batches and then use poll to send data to Redshift in intervals of time

Ответить
Ethan Quenum
Ethan Quenum - 30.03.2022 00:28

You must be french.

Ответить
chinmay shetye
chinmay shetye - 29.03.2022 04:21

nicee

Ответить
Ro
Ro - 30.12.2021 13:36

That's a kind of a lecture people are usually dreaming of! I'm just getting started with Kafka and your lessons have made my day: thank you <3

Ответить
yun lin
yun lin - 18.12.2021 13:14

When a consumer is to consume N records from Partition 0, 1, 2 of a topic, what does it mean? Is it reading N records from each partition? That would make it 3N records in total. Is it reading in round-robin fashion from partition 0, 1, 2, until the total read records is N?

Ответить
Satyaprakash Parida
Satyaprakash Parida - 25.08.2021 17:09

Thanks a lot. I am thinking that I got the best Kafka materials.

Ответить
Performance Testing Live Training, Pavan, MGM IT
Performance Testing Live Training, Pavan, MGM IT - 15.08.2021 09:13

can some one who is good in PPTs explain how those Message boxes are designed in power point?

Ответить
Valmik Kalathia
Valmik Kalathia - 29.06.2021 21:00

Thank you. It was a really nice explanation.

Ответить
Pradeep B
Pradeep B - 09.06.2021 13:04

How create dynamic topics

Ответить
meghana Pulavarthy
meghana Pulavarthy - 03.06.2021 15:34

How to fetch offset for particular events in the consumer??

Ответить
Didam Ns
Didam Ns - 28.05.2021 00:37

Merci beaucoup. Very clear.

Ответить
Andrew Kondrashov
Andrew Kondrashov - 18.01.2021 20:36

What for i have to use partitions?

Ответить
mason lu
mason lu - 09.01.2021 14:08

thanks, very helpful

Ответить
Sreenivasa Madenahally
Sreenivasa Madenahally - 29.12.2020 08:59

Thank you, very well explained.

Ответить
Suman K
Suman K - 25.11.2020 02:44

Great Lecture , Thanks a lot !!!

Ответить