This content originally appeared on DEV Community and was authored by Pascal Matthiesen
Alright, log lovers, let's get this observability party started 🎉. We're about to turn your Cloudflare logs from wallflowers into the life of the data analysis party 🥳.
Challenge 😕: Cloudflare Logpush delivers valuable data, but it might not be in the most usable format for further analysis.
Solution 💡: Enter Vector and Loki, the dream team 🏆. We'll leverage Vector's built-in HTTP server to accept your Cloudflare log lines, process them, and then ship them off to Loki for storage and analysis 🚀.
Why Bother? 🤔
Find and fix issues faster: Your logs will be organized and searchable, making troubleshooting a breeze.
Optimize performance: Identify bottlenecks and fine-tune your applications like a pro.
Boost security: Detect threats and protect your digital assets.
Get Ready to Geek Out!
We'll dive into the technical details soon, so get your command line fingers ready. You'll learn how to:
- Set up Vector to transform and forward your logs.
- Get Loki ready to ingest and analyze your data.
- Configure Cloudflare Logpush to send logs via HTTP.
These commands give your Kubernetes cluster a VIP backstage pass 🎫 to the hottest logging tool in town: Vector! ✨
-
helm repo add vector https://helm.vector.dev: This tells Helm (your trusty Kubernetes package manager) where to find Vector's exclusive swag, like adding a secret app store to your phone 🤫. -
helm repo update: This command refreshes Helm's knowledge, making sure you have access to the freshest Vector releases. Think of it as checking for those "new app" notifications on your phone 📲.
helm repo add vector https://helm.vector.dev
helm repo update
This Kubernetes Secret ("vector") is the Fort Knox 🔒 of your Vector logging tool, guarding a super-secret HTTP password ("123abc") 🤫. But hold on, Captain Obvious here 🦸 – change that password before some sneaky cyber ninja 🥷 steals your precious logs!
apiVersion: v1
kind: Secret
metadata:
name: vector
namespace: vector
stringData:
HTTP_PASSWORD: "123abc"
This Vector configuration sets up a secure gateway 🚪 for your Cloudflare logs. It grabs the logs, translates them into a format Loki understands 🗣️, and then sends them off for safekeeping 📦. It's like a trusty butler for your website data 🤵.
envFrom:
- secretRef:
name: vector
service:
ports:
- name: http
port: 3000
enabled: true
role: Stateless-Aggregator
customConfig:
acknowledgements:
enabled: true
sources:
in:
type: "http"
address: 0.0.0.0:3000
strict_path: false
encoding: text
path: /ingest
auth:
username: cloudflare
password: "${HTTP_PASSWORD}"
transforms:
parse_json:
type: remap
inputs:
- "in"
source: |
. = parse_json!(.message)
.timestamp = from_unix_timestamp!(.EventTimestampMs, unit: "milliseconds")
sinks:
out:
type: "loki"
tenant_id: "0:0"
encoding:
codec: "json"
remove_timestamp: false
out_of_order_action: accept
labels:
job: cloudflare
endpoint: http://loki-gateway.monitoring.svc.cluster.local
inputs:
- "parse_json"
This command summons the Helm installer 🧙♂️ to conjure up Vector on your Kubernetes cluster:
helm install vector vector/vector \
--namespace vector \
--create-namespace \
--values values.yaml
It's like giving Helm a magic spell book 🪄, pointing to the Vector spell page, and telling it where to build Vector's cozy little home 🏡 (in the "vector" namespace) with the customizations you specified in the "values.yaml" file. Now you're ready to wrangle those logs! 🧹🪵
Now, let's teach Cloudflare to share those juicy logs with Vector! 🤝 Just replace the placeholders and run this command in your terminal:
curl -X POST "https://api.cloudflare.com/client/v4/accounts/<YOUR_CF_ACCOUNT_ID>/logpush/jobs" \
-H "Authorization: Bearer <YOUR_CF_API_TOKEN>" \
-H "Content-Type: application/json" \
-d '{
"name": "vector-logs",
"destination_conf": "<YOUR_VECTOR_ENDPOINT>?header_Authorization=Basic%20<BASE64_ENCODED_AUTH>",
"dataset": "workers_trace_events",
"enabled": true
}'
Replace these placeholders:
- : Your Cloudflare account ID 🔢
- : Your Cloudflare API token 🗝️
- : The full URL to your Vector's HTTP endpoint (e.g., https://vector.yourdomain.com/ingest) 🌐
- : Your Vector username and password encoded in Base64 format 🔒 (
echo -n "$username:$password" | base64)
That's it! Cloudflare will start pushing logs to Vector, and Vector will whisk them off to Loki for safekeeping and analysis. 💨 Now you're a log analysis wizard! 🧙♂️
And with that, the logs have been tamed... for now. 😈 But rest assured, they'll be back with a vengeance, just like my insatiable craving for pizza. 🍕 If you enjoyed this wild ride through log analysis, join me next time as I tackle more tech challenges, share hilarious startup fails, and maybe even reveal my secret pizza recipe (just kidding...or am I? 😉).
This content originally appeared on DEV Community and was authored by Pascal Matthiesen
Pascal Matthiesen | Sciencx (2025-08-05T07:38:47+00:00) Cloudflare Logs: Not Just for Breakfast Anymore 🥞. Retrieved from https://www.scien.cx/2025/08/05/cloudflare-logs-not-just-for-breakfast-anymore-%f0%9f%a5%9e/
Please log in to upload a file.
There are no updates yet.
Click the Upload button above to add an update.