[Feature] Add user-defined precision in data table #271

New Issue

saeckech · 2024-05-02T06:42:03Z

saeckech commented

2024-05-02 06:42:03 +00:00

Current behavior

I have a temperature log which I want to plot and store alongside some data. I converted the human-readable timestamps into unix timestamps, which for a precision of 1 second have 10 digits (until near the end of the year 2286). The auswerteprogramm can correctly read and display the 10-digit numbers, but apparently stores and exports them with only 8-digit precision. Marking values displays the marks for datapoints with identical leading 8 digits on the same x-value (see screenshot).

Version

2024-04-30

Expected behavior

enough digits to accomodate numerical timestamps with second-precision or higher

Steps to reproduce

attached a temperature log as dummy data

Log messages

No response

Anything else?

Cannot tell if the 'sort values' function also only considers the first 8 digits.
Bonus points if when exporting as .agr, the grace-precision is automatically increased above the default 8 digits as well.

### Current behavior I have a temperature log which I want to plot and store alongside some data. I converted the human-readable timestamps into unix timestamps, which for a precision of 1 second have 10 digits (until near the end of the year 2286). The auswerteprogramm can correctly read and display the 10-digit numbers, but apparently stores and exports them with only 8-digit precision. Marking values displays the marks for datapoints with identical leading 8 digits on the same x-value (see screenshot). ### Version 2024-04-30 ### Expected behavior enough digits to accomodate numerical timestamps with second-precision or higher ### Steps to reproduce attached a temperature log as dummy data ### Log messages _No response_ ### Anything else? Cannot tell if the 'sort values' function also only considers the first 8 digits. Bonus points if when exporting as .agr, the grace-precision is automatically increased above the default 8 digits as well.

lakeshore.log.2024-04-23.csv

4.5 MiB

auswerteprogramm_precision.png

158 KiB

saeckech added the

Kind/Bug

label 2024-05-02 06:42:03 +00:00

dominik added

Kind/Enhancement

and removed

Kind/Bug

labels 2024-05-02 08:42:19 +00:00

dominik commented

2024-05-02 08:42:26 +00:00

Das ist ausschließlich in der Darstellung unter "Values", wo die Werte auf 8 Stellen beschränkt sind, ein Problem, es wird aber immer mit den kompletten Werten gearbeitet.
Die werden zumindest für mich auch mit mehr Stellen gespeichert, auch in der agr. Du musst da eben die Precision der Spalte erhöhen, daran kann ich absolut nichts ändern.

Das ist ausschließlich in der Darstellung unter "Values", wo die Werte auf 8 Stellen beschränkt sind, ein Problem, es wird aber immer mit den kompletten Werten gearbeitet. Die werden zumindest für mich auch mit mehr Stellen gespeichert, auch in der agr. Du musst da eben die Precision der Spalte erhöhen, daran kann ich absolut nichts ändern.

dominik self-assigned this 2024-05-02 08:42:32 +00:00

dominik changed title from ~~[BUG] internal precision too low for unix timestamp~~ to [Feature] Add user-defined precision in data table

2024-05-02 08:43:25 +00:00

dominik added this to the (deleted) project 2024-05-15 15:39:50 +00:00

Sign in to join this conversation.

2 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: IPKM/nmreval#271