top of page

Large Telemetry Counters Losing Single Counts? Check JSON Integer Precision

Nov 3
3 min read

A cumulative counter increases at the device, but its exported value sometimes stays unchanged or jumps by two. Before changing meter scaling, compare the exact source digits with every representation downstream. A numeric conversion can merge distinct large integers even when communications are working correctly.


Original conceptual illustration of counter digits travelling through a magnifying lens towards data storage

Original conceptual illustration; not a live ProSight Software screen, specific product or customer installation.


Key takeaways


  • Preserve the original exact digits before conversion.

  • Test adjacent values through every processing boundary.

  • Display formatting cannot recover precision already lost.


1. Separate transport success from numerical correctness


Receiving a valid message proves only part of the data path. A decoder, serializer, ingestion service, database field, query expression, browser or spreadsheet may change how a number is represented. An error introduced early can then look consistent everywhere else.


Start by recording the source data type, signedness, scale factor and engineering unit. A device's documented integer width does not prove that every receiving component preserves that width. Keep raw counter values separate from calculated consumption and from the formatted label an operator sees.


2. Locate the precision boundary


RFC 8259 permits implementations to limit numeric range and precision. Its interoperability guidance identifies integers from −(2^53)+1 to (2^53)−1 as values on which implementations using the discussed common numeric representation agree exactly. This is guidance about interchange; it is not a claim that JSON syntax itself has a universal 53-bit maximum.


The practical question is which representation each actual component uses. Some paths preserve larger integers exactly; others convert them to binary64 floating point. Test your deployed path rather than assuming the database type or gateway specification describes the whole chain.


3. Reproduce a one-count discrepancy


Consider the hypothetical exact readings 9007199254740992 and 9007199254740993. Their integer difference is one. Converting both to binary64 with ordinary nearest rounding can produce the same represented number, 9007199254740992, so subtracting those converted values produces zero.


These deliberately large test values illustrate a software boundary, not a specification for a named instrument. Many ordinary counters never approach this magnitude. High-resolution timestamps, accumulated fine-unit totals and long numeric identifiers deserve particular attention because their representation requirements may differ from a plotted engineering measurement.


A decimal-place setting changes the display. It cannot reconstruct a digit discarded during conversion. Equally, changing from pulses to kilolitres can hide the symptom while leaving exact reconciliation impossible.


Hypothetical binary64 conversion merges 9007199254740992 and 9007199254740993, turning their exact difference of one into zero

Original worked-example diagram. Values are illustrative and assumptions are explained in the text; this is not a product specification or field result.


4. Test the complete route


Use an isolated test dataset with consecutive integers below, around and above the relevant representation boundary. Keep the original values in a form that cannot be silently rounded by the tool used to record the test.


  1. Inject known consecutive values without connecting the test to control actions.

  2. Compare exact raw input with decoded and serialised values.

  3. Read stored values through the intended query and export paths.

  4. Check both the displayed total and its independently calculated difference.

  5. Record the first boundary where exact agreement is lost.


5. Select a representation deliberately


Possible designs include an exact integer type across the supported path, a documented decimal-string representation with explicit parsing, or separately managed raw counts and derived engineering values. A string is not automatically a drop-in numeric replacement: sorting, validation, queries and arithmetic must all follow the agreed contract.


For identifiers that are never quantities, preserving the identifier as text may be appropriate. For consumption calculations, decide where exact subtraction occurs and where rounding is permitted. Record units and precision alongside the design; do not ask an operator to infer them from a chart.


An acceptable fix preserves intended behaviour in exports as well as live views. Check CSV handling and spreadsheet import separately, because a correct stored value can still be altered when opened for analysis.


6. Agree the acceptance evidence


Keep a small before-and-after record containing original digits, each conversion boundary, calculated differences and the final export. Include a failing case and a nearby passing case. Repeat it after changes to the decoder or integration software.


This is an integration acceptance method, not a promise that any particular ProSight Software deployment supports every numeric type. Confirm the available ingestion, storage, calculation and export behaviour for your project. For billing or consequential operational decisions, agree accuracy and validation requirements with the responsible people before relying on derived totals.


Related guide and project resources




Technical references



Need help with your application?


Send a representative dataset, equipment details and the calculation or reporting rule you need to verify. Confirm the supported functions and integration requirements for your actual deployment.










Comments


prosight-software-logo (1).png
bottom of page