On Mon, 16 Sep 2019 06:06:25 +0000 René Romijn Rene.Romijn@tabsholland.nl wrote:
We did an upgrade to a new VM, we installed Debian Buster and we installed NAV 4.9.8. After that we did a restore of the postgress db with navsyncdb, and we also restored the /var/lib/graphite directory to have the history of the graphs.
But now if i reboot the new NAV server, the graphs are resetted tot he point we did the restore.
You mean your old time series data appears to be missing, even after a restore of the data files? That just seems strange. Either your carbon-cache has overwritten the files, which would be a serious bug, or your carbon-cache and/or graphite-web installations are not configured to use the directory where you restored your files.
Are you able to confirm that the files in use have the old data in them (e.g. using whisper-dump), and that their timestamps are being updated as carbon-cache writes to them?
Other than that, there's probably more knowledgeable support for Graphite issues if you ask questions in the Graphite project's preferred forums: [1].
File "/usr/lib/python3/dist-packages/carbon/cache.py", line 116, in choose_item return next(self.queue) builtins.StopIteration:
Is this easy to fix?
I have no idea, and I'm not sure how this would tie in with graphs appearing to be reset. This looks very much like this bug report from the graphite-project's issue tracker: [2]
Debian Buster features graphite-carbon version 1.1.4, but the bug was fixed in 1.1.5. I see a bug report [3] has been filed with Debian, as this bug causes a hard crash, but there's no fix yet. It might help to push the package maintainer along by adding your own feedback on [3].
Unless this is fixed, we'll probably run into this problem as we begin upgrading our production platforms to Buster as well :-P
[1] https://graphite.readthedocs.io/en/latest/install.html?highlight=help#help-i... [2] https://github.com/graphite-project/carbon/issues/815 [3] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=923464