EFK Setup in Kubernetes Cluster | Logging in Kubernetes with Elasticsearch Fluentd and Kibana | ELK

Nice Video, it Very useful for Kubernetes environment to check the logs

Can you please help me to perform same setup on AKS.. I look forward for your response..

I got error [in_tail_container_logs] pattern not match: "{\"log\":\"\\\\......} when running command: kubectl logs pod/fluentd-.....
Can you help me ?

Short and to the point. I will give this a try. Thanks for the effort.

hi,
can u tell me how can we load the logs of all namespaces in a single kibana dashboard

Thank you so much for making this content especially for the installation of Elastic Search,

I am assuming to persist the logs in elastic search, you would have to add pv so that kibana can show historical logs even if pods are restarted or deleted. Am i correct?

Bro I am new to kubernates..we have existing kubernates cluster with Linux worker node I am trying to setup efk for these but I am unable to provision statefulset and kibana deployment using docker image but nginx image is working and provisioning both but the major problem here is I am not able to run kubectl port forward command it's getting error

When tried to port forward kibana its lyk kibana server not yet ready

When trying to port-forward svc/elasticsearch 9200 error connection refused

NICE BRO, SHORT, WORKLY , easy to make , thanku very much from Argentina :)

I have successfully set up everything and can retrieve the logs. However, the issue lies in obtaining logs directly from files within the container, rather than relying on 'kubectl logs.' Do you have any ideas or hints on how to achieve this?

Hey thank you for the great video. Everything is working, but when i try to access kibana on the browser, it keeps saying "Kibana server is not ready yet", even after a long time. Do you know how to fix this? Greets!

For those who are facing not match proper logs from fluentd and got "/////"
just remove
- name: FLUENT_CONTAINER_TAIL_PARSER_TYPE
value: /^(?<time>.+) (?<stream>stdout|stderr)( (?<logtag>.))? (?<log>.*)$/
from daemonset.yaml

I really can't understand this fckn english... Damn!

Hi , it’s nice video can you please make a similar video for opensearch and opensearch dashboard since most of them are using that nowadays after docker runtime migration

i have followed same step, but getting below error
@jump-host:~/log/efk-setup/fluentd$ kubectl get pod
NAME READY STATUS RESTARTS AGE
es-cluster-0 1/1 Running 0 19m
es-cluster-1 1/1 Running 0 18m
es-cluster-2 1/1 Running 0 18m
fluentd-7qdbv 0/1 CrashLoopBackOff 7 (3m42s ago) 14m
kibana-85757f568c-xhgdv 1/1 Running 0 17m
@jump-host:~/log/efk-setup/fluentd$ kubectl logs fluentd-7qdbv
exec /usr/local/bin/tini: exec format error
@jump-host:~/log/efk-setup/fluentd$

good work bhoopesh

i have a question, we have not installed logstash , but still why its is shown in kibana dashboard

hey wanted to ask something, the configmap and rbac which we are applying, how will the helm chart of fluentd know about it, like where do we mention these files in the values.yaml of fluentd??

I have deployed five different microservices in a Kubernetes cluster as deployments. I have also configured Filebeat on my worker nodes. Now, I need to create an index in Kibana for each microservice. Could you please provide me with the Filebeat configuration file for this? I am using AWS EKS.

I am only getting default index as filebeat-<date>

Can anyone make a video for deployment of elk+ filebeat using Kubernetes with ssl enabled

how to send kubernetes logs to s3

Hi, Thank you for video. I have question about it: How do you to delete log ElasticSearch by automation after 7days? So it easy make full disk?

can we give public ip instead of local host. i gave but i got this error while executing systemctl enable command "unable to resolve host ip-172-31-59-93: Temporary failure in name resolution"

thanks man for this