pic_array_hash Module

A 32- and 64-bit FNV-1a state hash over arrays of intrinsic type.

pic_hash_32bit_fnv already hashes rank-1 integer arrays and character strings. This module fills the gap needed to fingerprint the state of a running program: real, complex and logical data, ranks 1 to 3, and a streaming accumulator so that many arrays – for example every component of a derived type – fold into a single digest.

Quick start

use pic_array_hash, only: array_hash, array_hash_t, array_hash_hex

integer(int32) :: fingerprint
type(array_hash_t) :: state

fingerprint = array_hash(my_matrix)             ! one shot

call state%update(sim%positions)                ! streaming
call state%update(sim%velocities)
call state%update(sim%step_count)
fingerprint = state%digest()
call logger%info("state " // array_hash_hex(fingerprint))

Supported data

• integer of kind int8, int16, int32, int64 – and therefore default_int, whichever width this build uses – ranks 0 to 3.
• real of kind sp and dp, ranks 0 to 3.
• complex of kind sp and dp, ranks 0 to 3.
• default logical, ranks 0 to 3.
• default character: a scalar string, and a rank-1 array of strings.

Scalars (rank 0) are reachable through update only; the one-shot array_hash takes arrays and strings. Rank 4 and higher, and real(qp), are deliberately not supported; see “Limits” below.

Byte stream definition (stable, and documented on purpose)

The digest is plain FNV-1a – 32-bit with offset basis 0x811C9DC5 and prime 0x01000193, 64-bit with offset basis 0xCBF29CE484222325 and prime 0x100000001B3 – over a byte stream built as follows. The stream itself is the same at both widths: everything below describes one definition, and the two digests differ only in the FNV parameters folded over it. Nothing in the stream depends on host endianness or on the storage layout of any type, so the same data gives the same digest on every machine and every supported compiler.

• An integer of encoded width w bytes contributes its w two’s-complement bytes, least significant byte first.
• A logical contributes one byte: 1 for true, 0 for false.
• A character contributes one byte per character, the iachar (ASCII) code of that character.
• A real contributes a fixed 14-byte canonical record: a 1-byte class code, a 1-byte sign, a 4-byte binary exponent and an 8-byte mantissa, each little endian. See the floating point notes below.
• A complex contributes the 14-byte record of its real part followed by the 14-byte record of its imaginary part.
• An array contributes the encoding of each element in Fortran array element order (column major), with nothing in between.

Floating point: what happens to -0.0, NaN and Inf

This is the part that bites checkpoint validators, so it is specified rather than inherited from whatever the bit pattern happens to be. A real value is not hashed by its bits. It is classified first, then canonicalised:

• Positive and negative zero hash identically. -0.0 and +0.0 compare equal in Fortran, so a validator built on raw bits would report a spurious mismatch between two runs that actually agree. Both are encoded as the single class code “zero”, with sign, exponent and mantissa all zero.
• Every NaN hashes identically. All NaNs – quiet or signalling, of either sign, with any payload – collapse to one class code “NaN”. Two checkpoints that both went NaN in the same place therefore agree. Note the consequence: a NaN never compares equal to itself numerically, but it does hash equal to itself here.
• +Inf and -Inf are distinguished from each other, and both are distinguished from every finite value and from NaN, because they carry separate class codes.
• Finite values are hashed exactly, with no rounding and no tolerance. The mantissa/exponent decomposition is injective on finite values, so two distinct finite numbers always produce distinct records. Subnormals are handled like any other finite value.

There is no raw bitwise entry point. Getting at the bits of a real requires transfer between real and integer kinds, which is precisely what this module avoids for portability; see “Portability” below.

Two consequences worth knowing:

• Because the hash is exact, it answers “are these bit-for-bit the same numbers”, not “are these the same to within a tolerance”. It is a reproducibility check, not a physics check.
• real(sp) and real(dp) holding the same finite numeric value hash differently, because the mantissa is scaled by the precision of the kind. Hash a checkpoint with the kind you stored it in. The zero, NaN and infinity records carry no mantissa, so those four values do hash the same in sp as in dp.

Shape sensitivity

The stream contains element values only; it does not contain the shape. A 2x3 array, a 3x2 array and a 6-element vector holding the same elements in the same order therefore all produce the same digest. That is the documented contract, and it is exactly what makes streaming equal to concatenation.

When shape must be part of the identity, feed it explicitly first:

call state%update(shape(a, kind=int32))   ! int32: build independent
call state%update(a)

Use kind=int32 there rather than the default, so the digest does not change between a default-integer build and a PIC_DEFAULT_INT8 build.

Empty arrays

A zero-sized array contributes no bytes at all. update on one is a no-op, and the one-shot array_hash of one returns ARRAY_HASH_OFFSET_BASIS. Every zero-sized array therefore has the same digest regardless of its type, kind or rank, and so does a zero-length string. If “empty” must differ from “absent” in your checkpoint, hash the size alongside the data as shown above.

Streaming, combining, and the concatenation contract

array_hash_t accumulates. Because the stream is a plain concatenation of per-element encodings, hashing a and then b into one accumulator gives exactly the digest of the concatenated array [a, b]. Sub-digests computed elsewhere – say, one per MPI rank – fold in as ordinary int32 data:

call state%update([rank_digest])

Order matters: FNV-1a is not commutative, so combine sub-digests in a fixed, agreed order.

Portability

No transfer between real and integer kinds anywhere; that is the one construct this module set out to avoid, because it behaves inconsistently on nvfortran and LFortran. Reals are taken apart arithmetically with the standard fraction, exponent, digits and scale intrinsics, which are exact for IEEE binary formats. Bytes are extracted with ibits, never with a shift by a negative count. The FNV multiply is carried out in int64 and reduced modulo 2**32 by masking; the 64-bit multiply, which has no wider kind to spill into, is carried out on 32-bit limbs instead (see fnv1a_byte64). Neither can overflow a signed integer. NaN and infinity are detected by comparison (unordered comparisons against zero, plus abs(x) > huge(x)) rather than through ieee_arithmetic, which LFortran does not fully provide.

That comparison-based classification assumes the compiler has not thrown IEEE semantics away. Building with -ffast-math / -Ofast, or nvfortran -Mfprelaxed, lets a compiler assume no NaN can occur and fold the NaN test away, after which a NaN would be decomposed as though it were finite. Do not build this module that way.

Limits

• Ranks 0 to 3 only. Rank 4 and up are reachable today by reshaping to a lower rank, which costs nothing in digest terms precisely because the hash is shape insensitive.
• real(qp) and complex(qp) are unsupported: a 113-bit mantissa does not fit the int64 mantissa field, and qp is not the same type on every target anyway.
• Only default logical and default character kinds.
• Neither width is a cryptographic checksum, and neither is collision resistant against an adversary. 32 bits is fine for comparing two digests of the same thing, where a false match has probability 2-32. It is not enough to identify things: among 105 distinct 32-bit digests the chance that some pair collides is about 69%, against 3e-10 at 64 bits. Use array_hash64 when digests are keys.