Data Consistency and Tradeoffs in Distributed Systems

What is the use of kafka if all the communication been hapening directly from Leader to follower ?

That 'C++' expression was priceless 😁

Amazing explanation!
Thanks a lot!

I feel more and more confident in system design daily... Thanks to you, sir!

The way you teach concepts from scratch/ bottom-up way is the most favourite thing in this channel !

Thanks for the video

I don’t think he released the next eventual consistency video

Amazing video. So easy to understand complicated topics. I have very limited experience with this and the pace and the content was perfect.

Do you know you don't need supercomputer to scale? You can scale with cloud computing. The best thing is that you don't need physical shopping. You just click and pay. 😜

The very quick smile you gave the camera when you said "c++" 🤣🤣🤣🤣

We are trying to solve single point of failure and consistency with distributed databases but ending up with the same . What am i missing here

you're a genious man

Gaurav, do the UK, US, and other countries' governments agree to store their data across other regions? what do you think?

Hi Gaurav, thanks for making such great contents regarding consistency in distributed systems. But in case of 2 Phase commit, if one of the follower node machine goes down, then the leader who got commit response from other follwers, also can't commit it and may need to rollup the transactions. So until and unless the single follower comes back, the system doesn't accept any write operations, so it still puts in a single point of failure right ?

He is wearing NITS Hacks T-shirt. I still remember when he came to NIT Silchar as a judge for NIT Hacks and we had a photo with him.

How did you solve the latency in a consistent system. Still servers have to send request from UK to EU for db updates (to remain consistent).

What happens if transaction is to add 100 rs and leader retried 100 times, so 100 rs added 100 times?

Tsunami in Harvard☺️

Thanks a lot for sharing this content! this is very helpful. Although I was aware of a lot of concepts there, your samples help me to figure them out a lot better. :)

For new readers, I think it should be explicitly pointed out that Consistency in ACID is very different from Consistency in CAP.
ACID consistency means that the values in DB should be valid i.e. if we are making two transactions both should complete before committing to disk.
CAP consistency means read requests to different nodes of a distributed system return the exact same data.
A system can exist without CAP consistency but a system should never exist without ACID consistency. ACID consistency is must-have for any RDBMS to be reliable/useful while CAP consistency can be a trade-off as per requirement.

To be honest C in ACID doesn't add any proper value. If a transaction is Atomic, Isolated, and Durable, it automatically becomes consistent.

What about partitioning? You didn't explain or I might have missed..

Is C in ACID properties o SQL same like CAP theorem? What u said is for CAP. But if u check the oracle Docs Consistency in ACID means Data should not violate any DB constraints FK, UK, Col types etc…correct me if am wrong.

Great explanation. I am having one questions as followed. Ideally, the followers or the replicas should reside in different regions or data centers to ensure availability. So, this introduces latency in every write operation in transforming the data synchronously from leader to the followers if we follow 2 phase locking. How to handle this ?

I like ur energy and the selfless appetite to share with community...

Hi Gaurav ,Thanks for such informative content. Looking forward for a series on design pattern

one more question

When we delete anything it store in our computer only in hard disk just remove the link so that we cannot find that file


And now we can store another data on this data it will overlap.

but



how can we transfer the deleted storage files to cloud that means our disc will can work for more years

how Inertnet work ? GB MA how jio airtel control net speed ?
Any new info ? apart from underground wires 😐

Hi Gaurav, just wondering whether high water mark index will help here and the client can be assured that the commit is acknowledged as anyways everything noways a async in nature and once all the nodes in the cluster are up to the speed we can start showing the change to the client.

Hi Gaurav Sen, when the video on Eventual consistency will come. Waiting for it

Thanks bro I have been doing front-end for the past 5 years. I’m just putting myself back on the market now and this video is a great refresher on the fundamentals consistency in distributed systems.

Now I get to know When we went to banks and they are like servers are not responding, but actually in the backend they are keeping themselves consistent. :D

Could you recommend any good system design concepts apart from your interview-focused course?

Next Hotstar 🙏... High in demand

So we can roll back messages from Kafka 🤔

This video is pure gold! Great stuff, Gaurav!

C >> C++ ... LOL

When i started listening to your videos ... i had no idea other e than you know your stuff, right now I can keep up quite alright...i now appreciate the level of your skill...and the chronological argument you give and use to explain. I love this man

I loved the pause on the C++ joke >_< as if to say yes I’m going there

Have a question.
If all the commit/prepares fail after multiple retries. Should we roll back on master as well? And in in 2 PC, instead of prepare and commit. Why can't we just: First send the update in phase 1 i.e. commit and then in phase 2 just compare A /master to slaves. If if matches it's done. If it fails we mark the state as disputed or eventual roll back

You said facebook is down and it was down after a week. what a coincident!

While rollback sent by the master, does it send to all slaves ?

Isn't the leader a SPOF here? Just curious.. may be if the leader fails, one of the followers become the leader? Wonder how that might happen and if there are databases that make it happen automatically.

Hi Gaurav, Great video! But I have a doubt. Why does communication over a long distance lead to latency? Don't servers use electromagnetic waves(speed of 3 x 10^8 m/s) to communicate with each other. So, the distance of a few 1000KM's will only cost few milliseconds. Is that enough to cause latency? Or did I miss something?

While I am watching this videos, Facebook, Instagram and WhatsApp are down

One thing I learnt from this video today is you can’t get super computers in super market.