pic_serialize Module

Self-describing binary serialization for PIC.

## Stream layout

Every stream starts with a 16 byte header followed by zero or more records. All multi-byte fields are written in the native byte order of the writing machine; the header carries a byte-order mark so that a reader on a foreign machine can detect this and un-swap on the fly.

### Header, 16 bytes

offset size field description
0 4 magic ASCII "PIC1", byte order independent
4 4 endian mark integer(int32) 0x01020304 (16909060)
8 4 version integer(int32) format version
12 4 reserved integer(int32) written as 0, readers skip it

There is deliberately no payload-length field in the header. The number and kind of records that follow is decided by the caller, and each record is individually length prefixed and bounds checked against the real size of the file, so a global length would only be a second source of truth that can disagree with the first. The reserved word exists so a future version can add one without changing the header size.

### Record, 12 byte record header plus payload

offset size field description
0 4 tag integer(int32) type tag, see PIC_TAG_*
4 8 count integer(int64) number of elements
12 n payload count * element size bytes

count counts elements, not bytes: three real(dp) values give count == 3 and a 24 byte payload. For a character record count is the number of characters, so the element size is one.

## Fixed widths on purpose

Every field that reaches the file is an explicit int32/int64, never default_int. A file written by a build with PIC_DEFAULT_INT8=ON is byte identical to one written by a default build; the test suite pins the exact byte length of a known stream so this cannot regress silently.

## Error contract

Every procedure takes type(error_t), intent(inout) :: err and writes it only on failure; it is never cleared on entry. Pass a fresh or cleared error_t and check it after each call, which is the convention the rest of PIC uses. Keeping err intent(inout) rather than intent(out) also keeps the compiler from emitting a conditional deallocation of the error message on every single call into this module.

## Usage

 type(error_t) :: err
 integer(default_int) :: unit
 integer(int32) :: version
 logical :: swapped
 real(dp), allocatable :: values(:)

 call serialize_open_write("state.bin", unit, err)
 call write_header(unit, err)
 call write_array(unit, [1.0_dp, 2.0_dp], err)
 call serialize_close(unit, err)

 call serialize_open_read("state.bin", unit, err)
 call read_header(unit, version, swapped, err)
 call read_array(unit, values, swapped, err)
 call serialize_close(unit, err)

Uses

  • module~~pic_serialize~~UsesGraph module~pic_serialize pic_serialize module~pic_error pic_error module~pic_serialize->module~pic_error module~pic_io pic_io module~pic_serialize->module~pic_io module~pic_types pic_types module~pic_serialize->module~pic_types module~pic_error->module~pic_types module~pic_global_definitions pic_global_definitions module~pic_error->module~pic_global_definitions module~pic_io->module~pic_types module~pic_ascii pic_ascii module~pic_io->module~pic_ascii iso_c_binding iso_c_binding module~pic_types->iso_c_binding iso_fortran_env iso_fortran_env module~pic_types->iso_fortran_env module~pic_ascii->module~pic_types module~pic_global_definitions->module~pic_types

Used by

  • module~~pic_serialize~~UsedByGraph module~pic_serialize pic_serialize module~pic_soa pic_soa module~pic_soa->module~pic_serialize module~pic_soa_particle pic_soa_particle module~pic_soa_particle->module~pic_soa

Variables

Type Visibility Attributes Name Initial
integer(kind=int32), public, parameter :: PIC_FORMAT_VERSION = 1_int32

Current envelope version. Readers accept 1 <= version <= PIC_FORMAT_VERSION and reject anything else with ERROR_VALIDATION.
integer(kind=int64), public, parameter :: PIC_HEADER_BYTES = 16_int64

Size of the stream header in bytes.
character(len=*), public, parameter :: PIC_MAGIC = "PIC1"

Four byte ASCII magic written at offset 0 of every stream. Being characters it has no byte order of its own, which is what makes it usable to reject a foreign file before anything else is decoded.
integer(kind=int64), public, parameter :: PIC_RECORD_HEADER_BYTES = 12_int64

Size of one record header, tag plus element count, in bytes.
integer(kind=int32), public, parameter :: PIC_TAG_CHAR = 5_int32

Record tag for a character scalar, 1 byte per element.
integer(kind=int32), public, parameter :: PIC_TAG_INT32 = 1_int32

Record tag for integer(int32) arrays, 4 bytes per element.
integer(kind=int32), public, parameter :: PIC_TAG_INT64 = 2_int32

Record tag for integer(int64) arrays, 8 bytes per element.
integer(kind=int32), public, parameter :: PIC_TAG_REAL_DP = 4_int32

Record tag for real(dp) arrays, 8 bytes per element.
integer(kind=int32), public, parameter :: PIC_TAG_REAL_SP = 3_int32

Record tag for real(sp) arrays, 4 bytes per element.
integer(kind=int32), private, parameter :: BITS_PER_BYTE = 8_int32

Width of one byte field, used by the byte reversal helpers.
integer(kind=int32), private, parameter :: PIC_ENDIAN_MARK = 16909060_int32

0x01020304, spelled in decimal so that no BOZ literal appears in an initializer, since gfortran and ifx disagree about the kind of those. Written natively: a reader that sees the byte reversed value 0x04030201 knows the file came from the opposite byte order.
integer(kind=int32), private, parameter :: RESERVED_WORD = 0_int32

Value written into the reserved header slot.

Interfaces

public interface read_array

Read one record back, validating its type tag and its length.

Usage: call read_array(unit, values, swapped, err)

values is allocatable and is allocated to the length stored in the record. swapped is the flag returned by read_header; when it is true every multi-byte element is byte reversed after reading.

Fails with ERROR_VALIDATION if the record holds a different type than the one requested, and with ERROR_IO if the payload announced by the record header runs past the end of the file.

  • private subroutine read_int32_array(unit, values, swapped, err)

    Read an integer(int32) array record.

    Arguments

    Type IntentOptional Attributes Name
    integer(kind=default_int), intent(in) :: unit
    integer(kind=int32), intent(out), allocatable :: values(:)
    logical, intent(in) :: swapped
    type(error_t), intent(inout) :: err

  • private subroutine read_int64_array(unit, values, swapped, err)

    Read an integer(int64) array record.

    Arguments

    Type IntentOptional Attributes Name
    integer(kind=default_int), intent(in) :: unit
    integer(kind=int64), intent(out), allocatable :: values(:)
    logical, intent(in) :: swapped
    type(error_t), intent(inout) :: err

  • private subroutine read_real_sp_array(unit, values, swapped, err)

    Read a real(sp) array record.

    On the native path the values are read straight into values. On the foreign endian path the same bytes are read into an int32 buffer, reversed with integer arithmetic, and only then reinterpreted.

    Arguments

    Type IntentOptional Attributes Name
    integer(kind=default_int), intent(in) :: unit
    real(kind=sp), intent(out), allocatable :: values(:)
    logical, intent(in) :: swapped
    type(error_t), intent(inout) :: err

  • private subroutine read_real_dp_array(unit, values, swapped, err)

    Read a real(dp) array record. See read_real_sp_array for the way the foreign endian path is kept away from the native one.

    Arguments

    Type IntentOptional Attributes Name
    integer(kind=default_int), intent(in) :: unit
    real(kind=dp), intent(out), allocatable :: values(:)
    logical, intent(in) :: swapped
    type(error_t), intent(inout) :: err

  • private subroutine read_char_record(unit, text, swapped, err)

    Read a character scalar record into a deferred length string.

    Characters are single bytes so swapped never changes the payload, but it is still needed to decode the record header itself.

    Arguments

    Type IntentOptional Attributes Name
    integer(kind=default_int), intent(in) :: unit
    character(len=:), intent(out), allocatable :: text
    logical, intent(in) :: swapped
    type(error_t), intent(inout) :: err

public interface write_array

Write one length prefixed, type tagged record to a stream unit.

Usage: call write_array(unit, values, err)

The unit must have been opened with access='stream' and form='unformatted', for instance by serialize_open_write, and write_header must already have been called on it. Zero sized arrays and zero length strings are legal and produce a record header with count == 0 and no payload.

  • private subroutine write_int32_array(unit, values, err)

    Write an integer(int32) array record.

    Arguments

    Type IntentOptional Attributes Name
    integer(kind=default_int), intent(in) :: unit
    integer(kind=int32), intent(in) :: values(:)
    type(error_t), intent(inout) :: err

  • private subroutine write_int64_array(unit, values, err)

    Write an integer(int64) array record.

    Arguments

    Type IntentOptional Attributes Name
    integer(kind=default_int), intent(in) :: unit
    integer(kind=int64), intent(in) :: values(:)
    type(error_t), intent(inout) :: err

  • private subroutine write_real_sp_array(unit, values, err)

    Write a real(sp) array record.

    Arguments

    Type IntentOptional Attributes Name
    integer(kind=default_int), intent(in) :: unit
    real(kind=sp), intent(in) :: values(:)
    type(error_t), intent(inout) :: err

  • private subroutine write_real_dp_array(unit, values, err)

    Write a real(dp) array record.

    Arguments

    Type IntentOptional Attributes Name
    integer(kind=default_int), intent(in) :: unit
    real(kind=dp), intent(in) :: values(:)
    type(error_t), intent(inout) :: err

  • private subroutine write_char_record(unit, text, err)

    Write a character scalar record. The stored count is the number of characters, so a record of n characters occupies n payload bytes.

    Arguments

    Type IntentOptional Attributes Name
    integer(kind=default_int), intent(in) :: unit
    character(len=*), intent(in) :: text
    type(error_t), intent(inout) :: err

Functions

public pure function host_is_little_endian() result(is_little)

Report whether the running machine stores the least significant byte of an integer first.

Read more…

Arguments

None

Return Value logical

public pure function record_bytes(tag, count) result(nbytes)

Total on disk size of a record, including its 12 byte record header.

Read more…

Arguments

Type IntentOptional Attributes Name
integer(kind=int32), intent(in) :: tag

One of the PIC_TAG_* constants.
integer(kind=int64), intent(in) :: count

Number of elements in the record.

Return Value integer(kind=int64)

private pure function bits_to_real_dp(bits) result(x)

Reinterpret the bit pattern of an int64 as a real(dp). See bits_to_real_sp for why this is confined to the swapped path.

Arguments

Type IntentOptional Attributes Name
integer(kind=int64), intent(in) :: bits

Return Value real(kind=dp)

private pure function bits_to_real_sp(bits) result(x)

Reinterpret the bit pattern of an int32 as a real(sp).

Read more…

Arguments

Type IntentOptional Attributes Name
integer(kind=int32), intent(in) :: bits

Return Value real(kind=sp)

private pure function bswap_int32(x) result(y)

Reverse the four bytes of an int32.

Read more…

Arguments

Type IntentOptional Attributes Name
integer(kind=int32), intent(in) :: x

Return Value integer(kind=int32)

private pure function bswap_int64(x) result(y)

Reverse the eight bytes of an int64.

Arguments

Type IntentOptional Attributes Name
integer(kind=int64), intent(in) :: x

Return Value integer(kind=int64)

private pure function element_bytes(tag) result(nbytes)

Bytes occupied by one element of the record type tag, or -1 for an unknown tag.

Arguments

Type IntentOptional Attributes Name
integer(kind=int32), intent(in) :: tag

Return Value integer(kind=int64)


Subroutines

public subroutine read_header(unit, version, swapped, err)

Read and validate the stream header.

Read more…

Arguments

Type IntentOptional Attributes Name
integer(kind=default_int), intent(in) :: unit

Stream unit opened for unformatted stream reading.
integer(kind=int32), intent(out) :: version

Format version found in the file, 0 if the header was rejected.
logical, intent(out) :: swapped

True when the file was written by a machine of the opposite byte order, so every multi-byte field needs reversing on the way in.
type(error_t), intent(inout) :: err

Error state.

public subroutine serialize_close(unit, err)

Close a unit previously opened by one of the open helpers.

Arguments

Type IntentOptional Attributes Name
integer(kind=default_int), intent(in) :: unit

Unit to close.
type(error_t), intent(inout) :: err

Error state, ERROR_IO if the close failed.

public subroutine serialize_open_read(filename, unit, err)

Open an existing serialized stream for reading.

Arguments

Type IntentOptional Attributes Name
character(len=*), intent(in) :: filename

Path of the file to read.
integer(kind=default_int), intent(out) :: unit

Newly opened unit number, -1 when the open failed.
type(error_t), intent(inout) :: err

Error state, ERROR_IO if the file cannot be opened.

public subroutine serialize_open_write(filename, unit, err)

Open filename for writing a serialized stream, replacing any existing file.

Read more…

Arguments

Type IntentOptional Attributes Name
character(len=*), intent(in) :: filename

Path of the file to create.
integer(kind=default_int), intent(out) :: unit

Newly opened unit number, -1 when the open failed.
type(error_t), intent(inout) :: err

Error state, ERROR_IO if the file cannot be created.

public subroutine write_header(unit, err)

Write the 16 byte stream header at the current position.

Read more…

Arguments

Type IntentOptional Attributes Name
integer(kind=default_int), intent(in) :: unit

Stream unit opened for unformatted stream writing.
type(error_t), intent(inout) :: err

Error state, ERROR_IO if any field could not be written.

private subroutine check_io(ios, context, err)

Turn a non zero iostat into an ERROR_IO carrying the raw value. Every I/O statement in this module funnels its status through here so that the failure message always names both the operation and the iostat the runtime produced.

Arguments

Type IntentOptional Attributes Name
integer(kind=default_int), intent(in) :: ios
character(len=*), intent(in) :: context
type(error_t), intent(inout) :: err

private subroutine read_char_record(unit, text, swapped, err)

Read a character scalar record into a deferred length string.

Read more…

Arguments

Type IntentOptional Attributes Name
integer(kind=default_int), intent(in) :: unit
character(len=:), intent(out), allocatable :: text
logical, intent(in) :: swapped
type(error_t), intent(inout) :: err

private subroutine read_int32_array(unit, values, swapped, err)

Read an integer(int32) array record.

Arguments

Type IntentOptional Attributes Name
integer(kind=default_int), intent(in) :: unit
integer(kind=int32), intent(out), allocatable :: values(:)
logical, intent(in) :: swapped
type(error_t), intent(inout) :: err

private subroutine read_int64_array(unit, values, swapped, err)

Read an integer(int64) array record.

Arguments

Type IntentOptional Attributes Name
integer(kind=default_int), intent(in) :: unit
integer(kind=int64), intent(out), allocatable :: values(:)
logical, intent(in) :: swapped
type(error_t), intent(inout) :: err

private subroutine read_real_dp_array(unit, values, swapped, err)

Read a real(dp) array record. See read_real_sp_array for the way the foreign endian path is kept away from the native one.

Arguments

Type IntentOptional Attributes Name
integer(kind=default_int), intent(in) :: unit
real(kind=dp), intent(out), allocatable :: values(:)
logical, intent(in) :: swapped
type(error_t), intent(inout) :: err

private subroutine read_real_sp_array(unit, values, swapped, err)

Read a real(sp) array record.

Read more…

Arguments

Type IntentOptional Attributes Name
integer(kind=default_int), intent(in) :: unit
real(kind=sp), intent(out), allocatable :: values(:)
logical, intent(in) :: swapped
type(error_t), intent(inout) :: err

private subroutine read_record_header(unit, expected_tag, swapped, count, ok, err)

Read a record header, check the type tag, and bounds check the announced payload against the real length of the file.

Read more…

Arguments

Type IntentOptional Attributes Name
integer(kind=default_int), intent(in) :: unit
integer(kind=int32), intent(in) :: expected_tag
logical, intent(in) :: swapped
integer(kind=int64), intent(out) :: count
logical, intent(out) :: ok
type(error_t), intent(inout) :: err

private subroutine write_char_record(unit, text, err)

Write a character scalar record. The stored count is the number of characters, so a record of n characters occupies n payload bytes.

Arguments

Type IntentOptional Attributes Name
integer(kind=default_int), intent(in) :: unit
character(len=*), intent(in) :: text
type(error_t), intent(inout) :: err

private subroutine write_int32_array(unit, values, err)

Write an integer(int32) array record.

Arguments

Type IntentOptional Attributes Name
integer(kind=default_int), intent(in) :: unit
integer(kind=int32), intent(in) :: values(:)
type(error_t), intent(inout) :: err

private subroutine write_int64_array(unit, values, err)

Write an integer(int64) array record.

Arguments

Type IntentOptional Attributes Name
integer(kind=default_int), intent(in) :: unit
integer(kind=int64), intent(in) :: values(:)
type(error_t), intent(inout) :: err

private subroutine write_real_dp_array(unit, values, err)

Write a real(dp) array record.

Arguments

Type IntentOptional Attributes Name
integer(kind=default_int), intent(in) :: unit
real(kind=dp), intent(in) :: values(:)
type(error_t), intent(inout) :: err

private subroutine write_real_sp_array(unit, values, err)

Write a real(sp) array record.

Arguments

Type IntentOptional Attributes Name
integer(kind=default_int), intent(in) :: unit
real(kind=sp), intent(in) :: values(:)
type(error_t), intent(inout) :: err

private subroutine write_record_header(unit, tag, count, ok, err)

Write a record header: type tag followed by the element count. ok rather than err carries the outcome so that a stale error left in err by an earlier call cannot be mistaken for a failure here.

Arguments

Type IntentOptional Attributes Name
integer(kind=default_int), intent(in) :: unit
integer(kind=int32), intent(in) :: tag
integer(kind=int64), intent(in) :: count
logical, intent(out) :: ok
type(error_t), intent(inout) :: err