Please be aware there has been a path change for the pve-exporter When I first set this up, the suggested volume mapping was /path/to/pve.yml:/etc/pve.yml That''s still working for me, although my container hasn't been updated recently After checking the dev's Github webpage, the suggested volume mapping is now /path/to/pve.yml:/etc/prometheus/pve.yml NOTE the extra prometheus folder in the container path
Good to hear I've used a lot of monitoring and alerting tools over the years but I'm really liking Prometheus and Grafana And although Proxmox VE does have some good monitoring of it's own, I think this combination is very useful
Thanks for making and sharing this video. I did some exporting of metrics in a different method a while back. Will have to lookup what I did and compare to you video to see the differences.
Thank you, great on sharing the rationale of your choices as each environment is different and it may not apply to each individual use case, so you taking the time to explain allows the end user to take an informed decision. Keep it up!
Thanks David, much appreciate and on my list. This weekend its guacamole docker to nginx proxy manager to cloudflare tunnel with email authentication. 👍
That's interesting. I didn't realized that we need to use this node exporter. I don't remember others creators mentioned it. I though in the end will be running only 2 container and not 3 (Exporter, Grafana and Prometheus). As always, thank you David!!!!
I've tried a lot of monitoring tools over the years but I do like how much easier this is to setup A backup and restore should be a lot easier too as I don't really need the data, I just need the docker compose and config files to get back up and runnning So far, the devs for the exporters I've been using provided good examples to at least get started and the community dashboards for grafana saved a lot of time SNMP can be a pain to setup if you don't already know the mibs but for now that's just going be needed for the physical switches
Hi David, thanks for the tutortials on monitoring. They are the best and most clear of all those I have watched and have gotten me set up on the first try. I am curious though, when running with a Proxmox cluster, how can we get the dashboard to better reflect the actual cluster stats i.e. the total amount of RAM. Instead of just showing the RAM of one node to show the total of all 3 nodes together. Or is it possible to show each node individually on a separate per-node dashboard?
I haven't customised the dashboard beyond the basics as I rely more on alerts If you look into the metrics provided you might find information there Though you could open multiple tabs I suppose for Grafana and select different nodes in each
Check the status of the containers to see if they are up and running, for example docker ps -a Also check the configs, for example, the IP addressing will probably be different for your network And check the logs of the containers to see what they're reporting, for example docker container logs node_exporter
Thank you! It was informative and very easy to follow. How can we apply this to a second pve server? Do we need to have a cluster or should we create another node-exporter and pve-exporter in the second pve server?
pve-exporter only needs to run on your docker server. It connects to the nodes over the network through an API Add each node you want to monitor to the prometheus job that you setup for prometheus to connect to the pve-exporter Bear in mind, when you have a cluster, pve-exporter will gather the same information from each node So you'll see the same details in Grafana But you still want to monitor all nodes in a cluster for redundancy reasons If you want details at a Linux level, then that's where node-exporter comes in But it needs to be installed on the host you want to monitor So if you want to use it to monitor your PVE nodes then install it on each node and configure prometheus to gather the data from each exporter
Nice video. Had some headaches around the new file path but got it working. Very nice dashboard with some very good metrics. One item to note: if you are using a drive point for your VM storage you will need give permissions to it as well. Also I had to use the username and password for the connection in the pve yaml file since I had multi Proxmox servers. Couldn't get the different token keys to work.
It's unusual for a path change like that It would be good if you could edit a video to make a slight change, like add some text, but the only option is to edit your own copy of the original file and upload it as a replacement I've tried a few monitoring solutions over time though and I do like this Prometheus and Grafana combo
@@TechTutorialsDavidMcKone Agree. It is a good centralized dashboard for monitoring all my Proxmox servers and able to get a quick over view. Love the simple but easy to read metrics to able to look for any potential issues.
@@cyberjohn44 Hi, John. I've been monitoring my Proxmox using influxDB, because, well, it has build in feature (metrix server). And I use dashboard 18621, but it lacking something, so I copy-paste from 15356, so I get its storage pool and modified it to using gauge. But, data from metric server is still missing something. 2 things. 1. temperature sensors. and I use telegraf on LXC to get CPU temperature. 2. Proxmox actual free memory, which is memfree - (cache+buff), which crucial in my opinion. is dashboard 10347 in the video have that (2 things)? is it possible to do that?
This is a great tutorial David! Easy to setup and get running but I'm not running a cluster, just 4 separate Proxmox servers so I can only monitor one of my servers. I googled and found some information about the "module" params but there is no official post anywhere on how set that up to monitor separate servers. Any ideas from anyone on how to accomplish that?
It might be easier to setup separate jobs/exporters as these are separate clusters in a way I was looking at the exporter documentation but there's no mention of separate deployments being pooled into one github.com/prometheus-pve/prometheus-pve-exporter
Excellent video, I used your tutorials on setting up Docker, Heimdall, Prometheus, and Grafana. I was attempt to add the ProxmoxVE monitor but I'm running into an issue with the docker container restarting after ~1minute. Any thoughts or tips on where to look?
Good to know the videos have been helpful If a container is restarting then check the container's log file, for example docker container logs prometheus It should tell you what the problem is and which line to check But chances are there's a typo in the script, even if it's just a space missing or an extra one added Or maybe it's pointing to something which doesn't exist e.g. a folder that needs mapping
Excellent video. I experienced a similar issue. This may help. I ran >docker compose up which enables me to see all the detail of the containers starting. I noticed this error under pve-exporter FileNotFoundError: [Errno 2] No such file or directory: '/etc/prometheus/pve.yml' I change the volume entry for pve-exporter to : volumes: - ./pve/pve.yml:/etc/prometheus/pve.yml This was successful. Good luck!
Hello David, generally great video, however I face the issue I don't get the standard metrics... I've only 5 PVE... Is there any explanation? Many thanks.
Hi David, thanks for the interesting video. I tried to follow your instructions, but I have issues to start the pve-exporter. I always get the error message: IsADirectoryError: [Errno 21] Is a directory: '/etc/pve.yml' . In which directory is the file pve.yml stored? Where is my error? Thanks for your support David.
Sounds like the mapping isn't lining up In the folder where the docker-compose.yml file is and where docker is being run from, I created a sub-folder called pve In that folder I created a file called pve.yml The docker compose file maps /etc/pve.yml to that file I created by referencing ./pve/pve.yml The period or dot is important because it references the current folder and it's a relative path i.e. relative from where docker is running from So the full line for mapping the config file is - ./pve/pve.yml:/etc/pve.yml Do check the syntax, which folder you're in and which folder you're running docker from Otherwise you could try an absolute path such as /home/docker/pve/pve.yml for instance if you are running docker from a home directory for a user called docker
For anyone that hits the issue with node-explorer container not starting due to error "pve-exporter | FileNotFoundError: [Errno 2] No such file or directory: '/etc/prometheus/pve.yml'" i did the following to fix it: -> edit docker-compose.yml and change pve-exporter volumes path from "/etc/pve.ym" to "/etc/prometheus/pve.ym". It should look as this: volumes: - './pve/pve.yml:/etc/prometheus/pve.yml'
I take it, you're monitoring PVE with node-exporter? I ask because you're referencing a folder called /etc/pve which I haven't referred to but would exist on a PVE server I didn't use that method in this video, instead I set up monitoring through the API to avoid software installation on PVE Although node-exporter would make sense for additional monitoring
Hi David!@@TechTutorialsDavidMcKone Yes, i followed your guide and got all the containers up yesterday but i got stuck later, at grafana not importing the dashboard profile. I will try to fix it later today. But, first of all: thank you so much for this guide, it helped me a lot, your brought me quite some time with this great guide. Regarding the error mentioned above, i hit it after configuring the node-explorer container (see instructions around minute 6:45 of your video): the external path is ./pve/pve.yml which your guide said it will map it internally to /etc/pve.yml. But in my case, with your config instructions, the pve-exporter container did not start and was complaining with the error "pve-exporter | FileNotFoundError: [Errno 2] No such file or directory: '/etc/prometheus/pve.yml' (docker compose logs). So, my workaround was to edit docker-compose.yml change the internal path from "/etc/pve.yml" to "/etc/prometheus/pve.yml"; the pve-exporter contaner ran without other issues after this change.
@@bogdandotct4863 That seemed odd as I checked my pve-exporter config and it's - ./pve/pve.yml:/etc/pve.yml I've had a look on the Github page and the path is now /path/to/pve.yml:/etc/prometheus/pve.yml So the devs must have changed the internal folder at some point, grrr Thanks for sharing as like others, I'll run into this myself the next time my container is updated
I did look into exporting metrics from Proxmox VE and I know it can send to InfluxDB and Graphite but I'm more familiar with pull models What puts me off push models is it requires punching a hole through the firewall to allow access to the monitoring server which I wouldn't been keen on from a security perspective I think Telegraf would make more sense than Prometheus for InfluxDB, although there is Prometheus exporter for it github.com/prometheus/influxdb_exporter
@@TechTutorialsDavidMcKone thank you. I have used Home Assistant integration that was pulling data from Proxmox directly to InfluxDB and then to Grafana
Show or tell file path (/home/username/xyz.abc) .. link to your doc/blog about the video then Chapters OR Chapters than link to blog before asking for money aka support is always better. @@TechTutorialsDavidMcKone
Very nice tutorial. Despite havin followed along, have been unable to get it working. prometheus exporter is able to scrape data from prox mox (can check the exporter pve target and see scraped metrics), but prometheus does not load the scrape_config for pve, nor do I see any pve endpoint target in prometheus.
I would suggest checking the config for Prometheus as it sounds like it isn't aware of the exporter It also needs to be reloaded if the config has been changed Otherwise it would be best to check the log file as any errors would show up there
@@TechTutorialsDavidMcKone Hey Dave, my bad, incorrectly mounted volume for prometheus. Going into running prometheus container and checking prometheus.yml showed me it was using the default, not my custom one from host. All good, thanks again!
Please be aware there has been a path change for the pve-exporter
When I first set this up, the suggested volume mapping was
/path/to/pve.yml:/etc/pve.yml
That''s still working for me, although my container hasn't been updated recently
After checking the dev's Github webpage, the suggested volume mapping is now
/path/to/pve.yml:/etc/prometheus/pve.yml
NOTE the extra prometheus folder in the container path
Thanks again for your great work! I'm up and running...
I am finding the combination of Prometheus and Grafana very useful
So good to know this video helped
thanks for the tutorial, it was really helpful! now i've got proxmox on my grafana dashboard via prometheus
Good to hear
I've used a lot of monitoring and alerting tools over the years but I'm really liking Prometheus and Grafana
And although Proxmox VE does have some good monitoring of it's own, I think this combination is very useful
agreed 100%!
Thanks for making and sharing this video. I did some exporting of metrics in a different method a while back. Will have to lookup what I did and compare to you video to see the differences.
Thanks for providing feedback, always appreciated
Great stuff! About to setup Grafana for the first time on my new Proxmox server, and this is going to help!
I like that it can be run in a container.
And having community dashboards makes setting this up a lot easier
Just what I have been looking for. Can't wait to watch.
Hope you find the video useful
Fairly straightforward to setup
Thank you, great on sharing the rationale of your choices as each environment is different and it may not apply to each individual use case, so you taking the time to explain allows the end user to take an informed decision. Keep it up!
Thanks for the feedback, really appreciated
Thanks David, much appreciate and on my list. This weekend its guacamole docker to nginx proxy manager to cloudflare tunnel with email authentication. 👍
Sounds like quite the weekend
Great video! Made this easy to setup.
I do like this combination for monitoring
So good to know the video was useful
That's interesting. I didn't realized that we need to use this node exporter. I don't remember others creators mentioned it. I though in the end will be running only 2 container and not 3 (Exporter, Grafana and Prometheus). As always, thank you David!!!!
There are other ways but I think this exporter is very useful
No agent needed on the nodes so less things to go wrong
Interesting, I monitor my proxmox using observium via snmp but this looks like a cool alternative to that setup with some advantages. Thank you
I've tried a lot of monitoring tools over the years but I do like how much easier this is to setup
A backup and restore should be a lot easier too as I don't really need the data, I just need the docker compose and config files to get back up and runnning
So far, the devs for the exporters I've been using provided good examples to at least get started and the community dashboards for grafana saved a lot of time
SNMP can be a pain to setup if you don't already know the mibs but for now that's just going be needed for the physical switches
Amazing, Would install in my HomeLab
Definitely a good open source monitoring solution and it's free to use
Thank you! This was very useful!
Good to hear the video was helpful
Great Video. Thank you!
Thanks for the feedback, always appreciated
Hi David, thanks for the tutortials on monitoring. They are the best and most clear of all those I have watched and have gotten me set up on the first try. I am curious though, when running with a Proxmox cluster, how can we get the dashboard to better reflect the actual cluster stats i.e. the total amount of RAM. Instead of just showing the RAM of one node to show the total of all 3 nodes together. Or is it possible to show each node individually on a separate per-node dashboard?
I haven't customised the dashboard beyond the basics as I rely more on alerts
If you look into the metrics provided you might find information there
Though you could open multiple tabs I suppose for Grafana and select different nodes in each
Thanks
Thank you very much
I followed these exact steps, but for some reason my prometheus only shows Prometheus up, it shows neither the nodes nor the exporter
Check the status of the containers to see if they are up and running, for example
docker ps -a
Also check the configs, for example, the IP addressing will probably be different for your network
And check the logs of the containers to see what they're reporting, for example
docker container logs node_exporter
Thank you! It was informative and very easy to follow. How can we apply this to a second pve server? Do we need to have a cluster or should we create another node-exporter and pve-exporter in the second pve server?
pve-exporter only needs to run on your docker server. It connects to the nodes over the network through an API
Add each node you want to monitor to the prometheus job that you setup for prometheus to connect to the pve-exporter
Bear in mind, when you have a cluster, pve-exporter will gather the same information from each node
So you'll see the same details in Grafana
But you still want to monitor all nodes in a cluster for redundancy reasons
If you want details at a Linux level, then that's where node-exporter comes in
But it needs to be installed on the host you want to monitor
So if you want to use it to monitor your PVE nodes then install it on each node and configure prometheus to gather the data from each exporter
Nice video. Had some headaches around the new file path but got it working. Very nice dashboard with some very good metrics. One item to note: if you are using a drive point for your VM storage you will need give permissions to it as well. Also I had to use the username and password for the connection in the pve yaml file since I had multi Proxmox servers. Couldn't get the different token keys to work.
It's unusual for a path change like that
It would be good if you could edit a video to make a slight change, like add some text, but the only option is to edit your own copy of the original file and upload it as a replacement
I've tried a few monitoring solutions over time though and I do like this Prometheus and Grafana combo
@@TechTutorialsDavidMcKone Agree. It is a good centralized dashboard for monitoring all my Proxmox servers and able to get a quick over view. Love the simple but easy to read metrics to able to look for any potential issues.
@@cyberjohn44 Hi, John.
I've been monitoring my Proxmox using influxDB, because, well, it has build in feature (metrix server). And I use dashboard 18621, but it lacking something, so I copy-paste from 15356, so I get its storage pool and modified it to using gauge.
But, data from metric server is still missing something. 2 things.
1. temperature sensors. and I use telegraf on LXC to get CPU temperature.
2. Proxmox actual free memory, which is memfree - (cache+buff), which crucial in my opinion.
is dashboard 10347 in the video have that (2 things)? is it possible to do that?
@@Anuitu2u Here are the 3 dashboards I am using. Node Exporter Full - 1860 Proxmox via Prometheus - 10347 Docker monitoring with node selection - 8321
This is a great tutorial David! Easy to setup and get running but I'm not running a cluster, just 4 separate Proxmox servers so I can only monitor one of my servers. I googled and found some information about the "module" params but there is no official post anywhere on how set that up to monitor separate servers. Any ideas from anyone on how to accomplish that?
It might be easier to setup separate jobs/exporters as these are separate clusters in a way
I was looking at the exporter documentation but there's no mention of separate deployments being pooled into one
github.com/prometheus-pve/prometheus-pve-exporter
@@TechTutorialsDavidMcKone That's what I figured but thought I'd ask.
Thank you!
Good to know the video was helpful
Excellent video, I used your tutorials on setting up Docker, Heimdall, Prometheus, and Grafana. I was attempt to add the ProxmoxVE monitor but I'm running into an issue with the docker container restarting after ~1minute. Any thoughts or tips on where to look?
Good to know the videos have been helpful
If a container is restarting then check the container's log file, for example
docker container logs prometheus
It should tell you what the problem is and which line to check
But chances are there's a typo in the script, even if it's just a space missing or an extra one added
Or maybe it's pointing to something which doesn't exist e.g. a folder that needs mapping
Excellent video.
I experienced a similar issue. This may help. I ran >docker compose up which enables me to see all the detail of the containers starting. I noticed this error under pve-exporter
FileNotFoundError: [Errno 2] No such file or directory: '/etc/prometheus/pve.yml'
I change the volume entry for pve-exporter to :
volumes:
- ./pve/pve.yml:/etc/prometheus/pve.yml
This was successful. Good luck!
Hello David, generally great video, however I face the issue I don't get the standard metrics... I've only 5 PVE... Is there any explanation? Many thanks.
It was an issue of misconfiguration. I didn't carefully follow the manual.
Good to know you've resolved this
Hi David, thanks for the interesting video. I tried to follow your instructions, but I have issues to start the pve-exporter. I always get the error message: IsADirectoryError: [Errno 21] Is a directory: '/etc/pve.yml' . In which directory is the file pve.yml stored? Where is my error?
Thanks for your support David.
Sounds like the mapping isn't lining up
In the folder where the docker-compose.yml file is and where docker is being run from, I created a sub-folder called pve
In that folder I created a file called pve.yml
The docker compose file maps /etc/pve.yml to that file I created by referencing ./pve/pve.yml
The period or dot is important because it references the current folder and it's a relative path i.e. relative from where docker is running from
So the full line for mapping the config file is
- ./pve/pve.yml:/etc/pve.yml
Do check the syntax, which folder you're in and which folder you're running docker from
Otherwise you could try an absolute path such as /home/docker/pve/pve.yml for instance if you are running docker from a home directory for a user called docker
Hello David,
thank you very much for your help. Now it works.@@TechTutorialsDavidMcKone
For anyone that hits the issue with node-explorer container not starting due to error "pve-exporter | FileNotFoundError: [Errno 2] No such file or directory: '/etc/prometheus/pve.yml'" i did the following to fix it:
-> edit docker-compose.yml and change pve-exporter volumes path from "/etc/pve.ym" to "/etc/prometheus/pve.ym". It should look as this:
volumes:
- './pve/pve.yml:/etc/prometheus/pve.yml'
I take it, you're monitoring PVE with node-exporter?
I ask because you're referencing a folder called /etc/pve which I haven't referred to but would exist on a PVE server
I didn't use that method in this video, instead I set up monitoring through the API to avoid software installation on PVE
Although node-exporter would make sense for additional monitoring
Hi David!@@TechTutorialsDavidMcKone Yes, i followed your guide and got all the containers up yesterday but i got stuck later, at grafana not importing the dashboard profile. I will try to fix it later today. But, first of all: thank you so much for this guide, it helped me a lot, your brought me quite some time with this great guide.
Regarding the error mentioned above, i hit it after configuring the node-explorer container (see instructions around minute 6:45 of your video): the external path is ./pve/pve.yml which your guide said it will map it internally to /etc/pve.yml. But in my case, with your config instructions, the pve-exporter container did not start and was complaining with the error "pve-exporter | FileNotFoundError: [Errno 2] No such file or directory: '/etc/prometheus/pve.yml' (docker compose logs). So, my workaround was to edit docker-compose.yml change the internal path from "/etc/pve.yml" to "/etc/prometheus/pve.yml"; the pve-exporter contaner ran without other issues after this change.
@@bogdandotct4863 That seemed odd as I checked my pve-exporter config and it's
- ./pve/pve.yml:/etc/pve.yml
I've had a look on the Github page and the path is now
/path/to/pve.yml:/etc/prometheus/pve.yml
So the devs must have changed the internal folder at some point, grrr
Thanks for sharing as like others, I'll run into this myself the next time my container is updated
I am very surprised that InfluxDB cannot be used ;-(
I did look into exporting metrics from Proxmox VE and I know it can send to InfluxDB and Graphite but I'm more familiar with pull models
What puts me off push models is it requires punching a hole through the firewall to allow access to the monitoring server which I wouldn't been keen on from a security perspective
I think Telegraf would make more sense than Prometheus for InfluxDB, although there is Prometheus exporter for it
github.com/prometheus/influxdb_exporter
@@TechTutorialsDavidMcKone thank you. I have used Home Assistant integration that was pulling data from Proxmox directly to InfluxDB and then to Grafana
This is a REALLY BAD how to video!! REDO IT
Can you be more specific please?
I need at least some points and suggestions
Show or tell file path (/home/username/xyz.abc)
..
link to your doc/blog about the video then
Chapters OR
Chapters than link to blog before asking for money aka support is always better.
@@TechTutorialsDavidMcKone
Very nice tutorial. Despite havin followed along, have been unable to get it working. prometheus exporter is able to scrape data from prox mox (can check the exporter pve target and see scraped metrics), but prometheus does not load the scrape_config for pve, nor do I see any pve endpoint target in prometheus.
I would suggest checking the config for Prometheus as it sounds like it isn't aware of the exporter
It also needs to be reloaded if the config has been changed
Otherwise it would be best to check the log file as any errors would show up there
@@TechTutorialsDavidMcKone Hey Dave, my bad, incorrectly mounted volume for prometheus. Going into running prometheus container and checking prometheus.yml showed me it was using the default, not my custom one from host. All good, thanks again!
@@TechTutorialsDavidMcKone oh, and subbed!