This content originally appeared on DEV Community and was authored by Arbythecoder
In this article, we will walk through setting up a centralized ELK (Elasticsearch, Logstash, Kibana) stack using Docker. This setup is crucial for monitoring and analyzing log data effectively.
Prerequisites
- Java: Ensure you have Java installed, as Elasticsearch requires it.
- Docker and Docker Compose: Install these tools for easier management of containerized applications.
Step 1: Clone the Repository
First, clone the repository where you want to set up your ELK stack:
git clone https://github.com/username/repo-name.git
cd repo-name
Step 2: Set Up Elasticsearch
-
Create a Docker Compose File:
Create a file named
docker-compose.ymlin the root of your project.
version: '3'
services:
elasticsearch:
image: elasticsearch:7.10.0
container_name: elasticsearch
environment:
- discovery.type=single-node
ports:
- "9200:9200"
- Start Elasticsearch: Run the following command to start the Elasticsearch service:
docker-compose up -d
Step 3: Set Up Logstash
-
Add Logstash to Docker Compose:
Update your
docker-compose.ymlfile to include Logstash.
logstash:
image: logstash:7.10.0
container_name: logstash
ports:
- "5044:5044"
volumes:
- ./logstash/conf:/usr/share/logstash/pipeline
- Create Logstash Configuration: Create a directory for Logstash configurations:
mkdir -p logstash/conf
Then create a file named logstash.conf inside that directory:
input {
beats {
port => 5044
}
}
output {
elasticsearch {
hosts => ["elasticsearch:9200"]
index => "logstash-%{+YYYY.MM.dd}"
}
}
Step 4: Set Up Kibana
-
Add Kibana to Docker Compose:
Update your
docker-compose.ymlto add Kibana.
kibana:
image: kibana:7.10.0
container_name: kibana
ports:
- "5601:5601"
Step 5: Run the ELK Stack
-
Start All Services:
Use the following command to start all services defined in your
docker-compose.yml:
docker-compose up -d
-
Access Kibana:
Open your web browser and navigate to
http://localhost:5601to access the Kibana dashboard.
Step 6: Configure GitHub Repository
- Initialize Git (if not already initialized):
git init
- Add Your Files:
git add docker-compose.yml
git add logstash/conf/logstash.conf
- Commit Changes:
git commit -m "Initial commit of ELK stack setup"
Create a GitHub Repository: Go to GitHub and create a new repository.
Add Remote Origin:
git remote add origin https://github.com/username/repo-name.git
- Push to GitHub:
git push -u origin master
Troubleshooting Guide
- Slow Docker Compose: Ensure Docker Desktop is running and restart if needed. Check system resources.
-
Elasticsearch Issues: Check logs with
docker-compose logs elasticsearchfor error messages. -
Kibana Access Problems: Ensure port
5601is free and not blocked by a firewall.
Conclusion
You have successfully set up a centralized ELK stack and pushed your configuration to GitHub. Use Kibana to monitor and analyze your log data effectively. If you encounter any issues, refer to the troubleshooting guide for assistance.
Feel free to customize any sections further!
This content originally appeared on DEV Community and was authored by Arbythecoder
Arbythecoder | Sciencx (2024-07-24T22:53:04+00:00) Day 18: Centralized ELK Stack Setup. Retrieved from https://www.scien.cx/2024/07/24/day-18-centralized-elk-stack-setup/
Please log in to upload a file.
There are no updates yet.
Click the Upload button above to add an update.