RabbitMQ on Kubernetes for beginners

RabbitMQ on Kubernetes for beginners

That DevOps Guy

54 года назад

44,935 Просмотров

Subscribe to show your support! https://goo.gl/1Ty1Q2 .
Patreon 👉🏽http://patreon.com/marceldempers

Today we're taking a look at running a RabbitMQ cluster on top of Kubernetes.
We run a simple one node kubernetes cluster, look at all the basic RabbitMQ concepts such as storage, persistence, authentication and discovery.
Checkout the source code below 👇🏽 and follow along 🤓

Also if you want to support the channel further, become a member 😎
https://marceldempers.dev/join

Checkout "That DevOps Community" too
https://marceldempers.dev/community

Source Code 🧐
--------------------------------------------------------------
https://github.com/marcel-dempers/docker-development-youtube-series

If you are new to Kubernetes, check out my getting started playlist on Kubernetes below :)

Kubernetes Guide for Beginners:
---------------------------------------------------
https://www.youtube.com/playlist?list=PLHq1uqvAteVvUEdqaBeMK2awVThNujwMd

Kubernetes Monitoring Guide:
-----------------------------------------------
https://www.youtube.com/playlist?list=PLHq1uqvAteVuEXCrRkPFWLXRKWNLOVUHn

Kubernetes Secret Management Guide:
--------------------------------------------------------------
https://www.youtube.com/playlist?list=PLHq1uqvAteVtq-NRX3yd1ziA_wJSBu3Oj

Like and Subscribe for more :)

Follow me on socials!
https://marceldempers.dev
Twitter | https://twitter.com/marceldempers
GitHub | https://github.com/marcel-dempers
Facebook | http://fb.me/thatdevopsguy
LinkedIn | https://www.linkedin.com/in/marceldempers
Instagram | https://www.instagram.com/thatdevopsguy

Music:
Track: jimmysquare - My Band | is licensed under a Creative Commons Attribution licence (https://creativecommons.org/licenses/by/3.0/)
Listen: https://soundcloud.com/jimmysquare/my-band

Track: Amine Maxwell - Cafe Cubano | is licensed under a Creative Commons Attribution licence (https://creativecommons.org/licenses/by/3.0/)
Listen: https://soundcloud.com/aminemaxwell/cafe-cubano

Track: Amine Maxwell - Memories | is licensed under a Creative Commons Attribution licence (https://creativecommons.org/licenses/by/3.0/)
Listen: https://soundcloud.com/aminemaxwell/memories

Track: Fox Beat 2 - Joakim Karud - Balloons - Royalty Free Vlog Music [BUY=FREE] | is licensed under a Creative Commons Attribution licence (https://creativecommons.org/licenses/by/3.0/)
Listen: https://soundcloud.com/foxbeatmusic2/joakim-karud-balloons-royalty-free-vlog-music-buyfree

Track: souKo - souKo - Parallel | is licensed under a Creative Commons Attribution licence (https://creativecommons.org/licenses/by/3.0/)
Listen: https://soundcloud.com/soukomusic/parallel

Тэги:

#devops #infrastructure #as #code #azure #aks #kubernetes #k8s #cloud #training #course #cloudnative #az #github #development #deployment #containers #docker #rabbitmq #messagequeues #messagebroker #messge #broker #queues #servicebus #aws #amazon #web #services #google #gcp
Ссылки и html тэги не поддерживаются


Комментарии:

@davoswain
@davoswain - 26.02.2021 11:21

Great video and series on RabbitMQ. I've followed your tutorial using GKE but also added an Ingress on port 80 with a domain name pointing to a loadBalancer pointing to port 1883 to receive MQTT messages. When I use the loadBalancer IP and port 80 to send a message it works but when I try and go through Ingress the message doesn't get through. The Python code sending the message doesn't throw an error. Just wondering if you have any words of wisdom to help resolve this issue? I want to be able to connect to the RabbitMQ Kubernetes cluster for all message connections using the Ingress and domanin name. Thanks.

Ответить
@vtvvnha2230
@vtvvnha2230 - 05.03.2021 13:12

awesome video

Ответить
@nabarundebjani
@nabarundebjani - 13.03.2021 17:28

Could you please provide the steps if ingress need to configure for management UI

Ответить
@davepazooki3891
@davepazooki3891 - 08.04.2021 22:07

Can I push rabbitmq:3-management docker image from my docker desktop to Azure, e.g. in a container register? If yes how? Thanks

Ответить
@concay8872
@concay8872 - 01.05.2021 01:35

Thank you so much. I tried to use your yaml script and I used persistence storage however, If I define the storage less than 500mb, it didn't work. CRASH REPORT Process <0.272.0> with 0 neighbours exited with reason: {{could_not_write_file,"/var/lib/rabbitmq/mnesia/[email protected]. so I increased them to 10Gi and worked. I don't understand what is the storage for? for the production what is the recommended size? what situation the storage will be full? thanks again

Ответить
@overthinker-tk1zn
@overthinker-tk1zn - 11.05.2021 12:49

Awesome ! Thank you!!

Ответить
@hcgaron
@hcgaron - 15.05.2021 06:38

Your videos are freaking amazing! I'm debating currently whether or not to run rabbitMQ in my k8s cluster or to go with a cloud SaaS solution. I can't tell what kind of Persistent Storage to use if I host it on my own. I'll probably deploy this to DigitalOcean, so I could use their storage. It sounds like using the actual droplet storage (node storage) would be bad because it won't be available across nodes, correct? What kind of storage do you recommend?

I'm doing a bit of cost analysis, since it sounds like I'll have to purchase some amount of storage - and it might be a wash to just use a cloud service and have it managed for me.

Thanks for all your great content!

Ответить
@magrag1987
@magrag1987 - 06.06.2021 16:39

hi, thnak you, i was looking for this lesson from longtime. in my organisation we have a cluster and i have some data loss, when i delete a pod. i am guessing this may be due to some policy, like HA-fed could be set to all nodes, and i want to check the status of the fed,
CAN YOU PLEASE HELP ME HOW TO CHECK THE FEDERATION-STATUS FROM A NODE, .. i used rabbitmqctl rabbit@NODE NAME federation-status but i didnt get any json file.
thank you

Ответить
@sumer9999
@sumer9999 - 06.07.2021 08:10

Gem of a video, life saver for many people

Ответить
@gianlucadivincenzo1956
@gianlucadivincenzo1956 - 06.07.2021 14:56

Simply <3

Ответить
@dave-ventura
@dave-ventura - 10.07.2021 21:04

stable upper arm! And a good tutorial, thanks

Ответить
@이찬진-o1g
@이찬진-o1g - 25.07.2021 11:31

i run m1 docker desktop k8s , kubectl get storageclass. and get name hostpath ... and change storageClassName : "hostpath"
it works! nice video
one question,,, is there any reason mirroring without rabbitmq-3 node?

Ответить
@brandonfang524
@brandonfang524 - 27.07.2021 05:57

Thanks for the great tutorial for RabbitMQ. I wondered if this also work without RBAC?
Cause we may not have namespace admin authority at work.

Ответить
@robwaldram1275
@robwaldram1275 - 22.08.2021 00:13

Because of you and @fireship I have been pushed to non-stop learn new skills. Right now slowly catching up to the main developers of the project I was an UX / UI designer in. Thanks for broadening my reality of what is possible. Keep up the amazing work!

Ответить
@abhilashhazarika6714
@abhilashhazarika6714 - 08.09.2021 01:46

Thank you for this. I am using this to deploy the cluster to AWS EKS. Can you please explain how to access the management dashboard of rabbitmq over nginx ingress load balancer?
Thanks. I have nginx setup that I can access other deployments but I am not sure how to configure nginx for statefulset.

Ответить
@yijencheng7641
@yijencheng7641 - 07.10.2021 09:16

Hi, thanks for your AMAZING tutorial. I just started understanding RabbitMQ clustering, and have went through your three videos. However, I'd like to know what's difference between using Kubernetes cluster versus the previous video(config file)? Cause using kubernetes seemed wayyyy more complex, but the two ways seemed to acheive the same outcome :( Thanks!

Ответить
@uhjarnaa
@uhjarnaa - 16.10.2021 17:08

Thank you very much for a very instructive video. I'm getting one issue with the publisher pod... Whenever I try to post a message to the pod it says connection refused. Any thoughts on what to check to get it working?
I have successfully test the publisher image when checking out the two other videos in the series.

Ответить
@RebeliousSapien
@RebeliousSapien - 29.10.2021 17:08

how is persistency maintained in Rabbitmq when it's Kubernetes ?? ... when you delete a rabbitmq pod, what happens to the queues items?

Ответить
@RebeliousSapien
@RebeliousSapien - 29.10.2021 19:21

do you still need the rbac and the plugin to find other rabbitmq instances if you only plan on having just the one ?

Ответить
@hectorfonseca1002
@hectorfonseca1002 - 20.11.2021 10:04

Thank you so much, you have positively impacted my life. Thank you!

Ответить
@kaustuvchatterjee4117
@kaustuvchatterjee4117 - 15.01.2022 17:33

Can you share the URL of the github repo you are referring?

Ответить
@kaustuvchatterjee4117
@kaustuvchatterjee4117 - 21.01.2022 01:39

How livenessprobe & readinessprobe can be implemented for rabbitMQ?

Ответить
@edmoretshuma8311
@edmoretshuma8311 - 26.05.2022 05:26

hie there , thanks for a very insightful video. I am getting an error with the erlang cookie when the containers startup : CRASH REPORT Process <0.200.0> with 0 neighbours crashed with reason: "Bad characters in cookie" in auth:init_no_setcookie/0. I set the cookie in bash with dd if=/dev/urandom bs=30 count=1 | base64 . Is there something else Im missing?

Ответить
@thatinstant
@thatinstant - 06.06.2022 04:22

This video falls apart when that devops guy adds mirroring by exec'ing into the pods instead of doing it right and ensuring that the mirroring setup is repeatable and declarative.

Ответить
@DominicTrix
@DominicTrix - 11.06.2022 21:13

Wow, your hard work setting these tutorials up is fantastic and appreciated. Dense amount of information but so well presented I feel like I had all my questions answered and more. Thank you.

Ответить
@isaacndarwa
@isaacndarwa - 26.07.2022 01:19

Thanks for the video! It's very helpful. I have one issue though: I can't get the port-forward to work. I am using a self-hosted cluster with OVH based servers, and the port-forward command just isn't working for me. Do you have any idea what the issue could be?

Ответить
@mzw8374
@mzw8374 - 07.09.2022 15:09

Hello, What if we setup the rabbitmq on EC2 instance not with containerization (k8s) for serving all the microservices on kubernetes cluster, is it possible?

Ответить
@roymollenkamp991
@roymollenkamp991 - 30.09.2022 20:15

Whatever you make... it's not enough. Thank you.

Ответить
@amitkumdixit
@amitkumdixit - 09.10.2022 18:16

port forwarding as to be done again one we delete any pods.

Ответить
@johnnanong8756
@johnnanong8756 - 14.10.2022 12:56

Any alternative than kind?
It seems like kind is very "go" dependent when installing.
Probably an alternative tool that you dont have to install other compiler?

Thanks!

Ответить
@helgardwagener
@helgardwagener - 23.10.2022 11:06

Not sure if you keep an eye on comments for old videos, this is invaluable thanks for explaining so clearly how to implement RMQ to K8S.

How would you update the RMQ config if the cluster was deployed via Helm?

Ответить
@babu_bangalore
@babu_bangalore - 02.02.2023 16:41

Followed all the steps, but the pods are in pending state, and it shows

Warning FailedScheduling 9m40s 0/5 nodes are available: 5 pod has unbound immediate PersistentVolumeClaims.
Warning FailedScheduling 9m40s 0/5 nodes are available: 5 pod has unbound immediate PersistentVolumeClaims.

What should I do ?

Ответить
@Sushic.
@Sushic. - 10.02.2023 11:50

Hi, I am encountered an error "Bad characters in cookie" while rabbitmq deployment, I am doing same things shown in video. But I can't deploy rabbitmq, could you help me?

Ответить
@jackmurak
@jackmurak - 17.02.2023 22:31

Beginners??? I think not

Ответить
@thangtrananh622
@thangtrananh622 - 16.03.2023 20:14

I have configured PVC to be 50Mi. It seems that it isn't enough. The rabbits usually crash. Then I decide to increase it to 5Gi, and everything is OK.

Ответить
@ananyasahoo8552
@ananyasahoo8552 - 10.05.2023 07:52

This video is really amazing and detailed. Have you made any videos regarding setting up rabbitmq in nomad? It would be really nice if you make one🙂

Ответить
@eniac78
@eniac78 - 01.10.2023 06:46

Thanks! I was following the example on the Rabbitmq website in the post entitled "Deploying RabbitMQ to Kubernetes: What's Involved?", but they left out the part about setting env vars for nodename, use long name, and k8s hostname suffix. I was about to give up and move back to operator until I concluded it was probably a dns name resolution issue and set those env vars from your example. Then it all clustered together! Thanks! :)

Ответить
@ashseth7885
@ashseth7885 - 02.11.2023 23:11

You are gem, thank you

Ответить
@patrickjustice7371
@patrickjustice7371 - 06.11.2023 17:13

mirrored queues are deprecated feature, you should use quorum queues for replicated queues

Ответить
@tynshjt
@tynshjt - 22.02.2024 10:34

RabbitMQ has Kubernetes Operator now. Do you recommend using that or this manual way is still better?

Ответить
@zheniachubarov3384
@zheniachubarov3384 - 19.05.2024 23:00

Hi! Thank you for the awesome video? But could you make an update because environment variable for erlang cookie is deprecated and newer versions use different approach

Ответить
@allinonevlog9169
@allinonevlog9169 - 25.07.2024 08:05

Hello sir, I followed all the steps you taught. But still i not getting rabbitmq dashboard.

Ответить
@NaorPeretz
@NaorPeretz - 11.09.2024 14:41

HI @MarcelDempers

few questions & requests for updates:
1. you use in post-forward command, after you delete rabbitmq-0, the UI management get lost and I need to re-run it, is that for you too? look like you don't run it again or you remove it in edit
2. why you don't show use in namespace inside the yaml files, once you move to complex systems with few namespaces, this is very useful.
3. add how you declare your queues on start the cluster inside the yaml files(probably more then 1 will be welcome), usually you will need more then 1 queue, at least for the hard stuff.
4. how you upgrade your versions? I found some information on it, but it didn't work me, and because I just create the cluster I don't need now, but it will be useful, Especially when the version you are showing has deprecated.
5 look like set_policy about the mirroring deprecated in 3.11.0 version, I think a lot of people will want too see it on the new version.

Thanks you:)

Ответить
@g3ar75
@g3ar75 - 08.11.2024 12:59

Really loved the previous 2 videos, this one was really nice too but next level for someone like me who rarely works with kubernetes.
Fault is at me for being a noob but I would love to know if you have any other kubernetes tutorials.

Final question, which would you recommend most? Minikube or Kind?

Ответить
@brockhard
@brockhard - 29.11.2024 21:26

Im curious. Is there a reason you would deploy it this way over using a helm chart? Ive really embraced gitops and can use argocd to deploy both ways. I wanna try it this way woth rabbit mq to see how it feels to manage it this way.

Would love to hear your opinion on managing deployments and when you would use helm or create the yml yourself.

Excited to see you making new videos. Youve been a major help learning kunernetes and pushed me to manage my own k3s cluster.

Ответить