@nazokiyoubinbou @Viss Exactly
-
@Viss I had similar concerns when I threw out netdata for Having Become Terrible(tm).
I didn't want to setup an elasticsearch cluster just to hold metrics.
It turns out grafana itself is fairly lightweight, and you can swap out prometheus server with something like VictoriaMetrics + vmagent, which lets you do push instead of pull depending on context, and also just normal endpoint scraping.
The datasource itself can either be transparently added as a prometheus one, or as a victoriametrics one (which nets you a few query language bonuses).
It's supremely low resource usage and does not consume much disk space either, to much of my surprise. None of it require a k8s cluster just to collect metrics either.
It's worth exploring, I think.
-
@Viss
SQL Server Reporting Services -
I don't have a firm idea of your options, but I wonder whether knowing more about your intended use case might help shape advice.
What kind of data are you looking to visualize, and how/where is it stored?
Is this for single user and as-needed, or something you might want running continuously or for multiple users?
I wonder if @hrbrmstr in particular has thoughts.
Like, depending on what you want to do, it might be as simple as throwing your data into DuckDB and generating some time series graphs with Plotly with a tiny bit of Python in a Jupyter NB, or a dashboard built using Dash. It's been a while since I've done any of that personally, though.
-
@Viss As long as you can push whatever metrics you have into a database/store of your choice, pretty sure you can use Grafana to visualize it.
I just push my metrics to postgres and query from Grafana.
-
@Viss I had similar concerns when I threw out netdata for Having Become Terrible(tm).
I didn't want to setup an elasticsearch cluster just to hold metrics.
It turns out grafana itself is fairly lightweight, and you can swap out prometheus server with something like VictoriaMetrics + vmagent, which lets you do push instead of pull depending on context, and also just normal endpoint scraping.
The datasource itself can either be transparently added as a prometheus one, or as a victoriametrics one (which nets you a few query language bonuses).
It's supremely low resource usage and does not consume much disk space either, to much of my surprise. None of it require a k8s cluster just to collect metrics either.
It's worth exploring, I think.
@Viss For reference also, it was all supremely trivial to configure. I put up a nuc-style PC with a N95 and 8gb of ram to hold the `graphs and shit` role in my homelab of like ~30 hosts, and it turns out that was WILDLY overspecced for the task. I don't know what kind of black magic VictoriaMetrics does in terms of storage and memory usage, but it's shockingly low.
-
Like, depending on what you want to do, it might be as simple as throwing your data into DuckDB and generating some time series graphs with Plotly with a tiny bit of Python in a Jupyter NB, or a dashboard built using Dash. It's been a while since I've done any of that personally, though.
@DaveMWilburn @hrbrmstr im building out orbital and i want graphs for it. i have a fairly robust queuing system and workers and api nodes that are passing jobs around back and forth and i need to see shit like how many jobs are in the queue, what the workers are doing, cpu/ram on the workers and api, etc.. in case i hit scaling issues early (which would be fucking epic), but i refuse to deal with the docker flavored grafana stack. its way way way topheavy
-
@Viss For reference also, it was all supremely trivial to configure. I put up a nuc-style PC with a N95 and 8gb of ram to hold the `graphs and shit` role in my homelab of like ~30 hosts, and it turns out that was WILDLY overspecced for the task. I don't know what kind of black magic VictoriaMetrics does in terms of storage and memory usage, but it's shockingly low.
@mr_daemon youre the second person to mention victoriametrics, and i went and looked at their site and their youtube page but i cant find, like, a simple quickstart guide, or screenshots of it, or a video of someone installing it or anything. does that even exist?
-
@Viss what does librenms use for vis? I always liked theirs
-
@Viss As long as you can push whatever metrics you have into a database/store of your choice, pretty sure you can use Grafana to visualize it.
I just push my metrics to postgres and query from Grafana.
@larsmb i wonder if theres a simple snmp connector for grafana that skips all that other crap
-
@Viss what does librenms use for vis? I always liked theirs
@h2onolan thats a good question - i have no idea. I need to build out some kinda monitoring/alerting system for orbital after this hetzner nonsense. i need to know the instant a scanning node goes away if some cloud host decides they dont ilke me anymore and doesnt say anything first.
-
@Viss Since InfluxDB added some graphing ability to their system in V2, I've not used Grafana.
I find I'm able to run the InfluxDB container, and then telegraf on the sources, and have a much smaller footprint.
-
@larsmb i wonder if theres a simple snmp connector for grafana that skips all that other crap
-
@DaveMWilburn @hrbrmstr im building out orbital and i want graphs for it. i have a fairly robust queuing system and workers and api nodes that are passing jobs around back and forth and i need to see shit like how many jobs are in the queue, what the workers are doing, cpu/ram on the workers and api, etc.. in case i hit scaling issues early (which would be fucking epic), but i refuse to deal with the docker flavored grafana stack. its way way way topheavy
@DaveMWilburn @hrbrmstr chiefly, basic system info. disk, ram, cpu, network usage at a minimum, if its got hooks to showme postres stats, that would be rad, bonus points for it can show me rabbitmq stats.
thats basically it.
i just need dashboard lights for my asm scanning platform so that when i do the big linkedin (urk) post announcing the promo and new pricing, and if i get a swell of new customers, i want to see that shit scale livebyobu is great for that, but no history
-
@mr_daemon youre the second person to mention victoriametrics, and i went and looked at their site and their youtube page but i cant find, like, a simple quickstart guide, or screenshots of it, or a video of someone installing it or anything. does that even exist?
@Viss
I believe they just assume everyone will just use grafana.
As far as working with vicoriametrics, the query syntax is very much like Prometheus, and concepts are similar.
@mr_daemon -
@mr_daemon sweet! thanks man!
-
@Viss
I believe they just assume everyone will just use grafana.
As far as working with vicoriametrics, the query syntax is very much like Prometheus, and concepts are similar.
@mr_daemon -
@mr_daemon @viq the more middleware i can skip the better. so i can light up victoriametrics, and that will squirt data directly into grafana?
-
@mr_daemon
But also similarly you can set up Prometheus directly on a system, without any containers anywhere.
@Viss -
@mr_daemon @viq the more middleware i can skip the better. so i can light up victoriametrics, and that will squirt data directly into grafana?
@Viss @viq Yeah, or rather, you add the VictoriaMetrics server as a data source in grafana and it will read the timeseries from it. You can setup the data source __twice__ as prometheus AND victoriametrics (via the plug in) if you want, which lets you both do graphql and also just straight up use generic prometheus dashboards.
-
@mr_daemon @viq the more middleware i can skip the better. so i can light up victoriametrics, and that will squirt data directly into grafana?