dimitri/bqn-npy
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
Serialize and deserialize arrays in BQN to the Numpy NPY format
8 commits 1 branch 0 tags 38 KiB
Find a file
Dimitri Lozeve 9a1d251716 Fix issue with unsigned integers 
Fixes https://github.com/dlozeve/bqn-npy/issues/1.

The spec says that •bit._cast should support 32-bit uints [1], but
CBQN does not [2].

To get around this, we just use signed integers, which works for
numbers smaller than 2^31. Above that, it will wrap around
and (probably) fail, but CBQN does not support them anyway (except by
converting them to doubles).

From Marshall:

> All of the provided functions at this time are identical on i32 and
> u32, because they work (mod 2^32) and the only difference between
> those formats is to add or subtract 2^32 from numbers outside the
> range [0,2^31).
> If you want to run on u32s stored directly as numbers in BQN (they
> won't fit in i32 and will be stored as double!) you can •bit._cast
> those into a signed format at the beginning and then back out at the
> end.

[1] https://mlochbaum.github.io/BQN/spec/system.html#bitwise-operations
[2] https://matrix.to/#/!EjsgbQQNuTfHXQoiax:matrix.org/$PL2AYNlLQuYhPftdt7yw7rW9heiVEEeATte2zOt2BYk
2024-11-11 22:58:04 +01:00
.gitignore Initial commit 2023-10-04 13:26:25 +02:00
gentest.py Fix issue with unsigned integers 2024-11-11 22:58:04 +01:00
LICENSE Add license and readme 2023-10-04 17:23:55 +02:00
npy.bqn Fix issue with unsigned integers 2024-11-11 22:58:04 +01:00
README.org Add license and readme 2023-10-04 17:23:55 +02:00
tests.bqn Fix issue with unsigned integers 2024-11-11 22:58:04 +01:00

README.org

Serialization and deserialization library for BQN arrays

This library allows to save and load BQN arrays to the binary NPY format compatible with Numpy. It can also be used to interoperate with Numpy.

Limitations

The library only support homogeneous multidimensional arrays with the following Numpy dtypes:

dtype Description
<f8 little-endian 64-bit double-precision floating point number
<i4 little-endian 32-bit signed integer
<u4 little-endian 32-bit unsigned integer

All other dtypes (big-endian numbers, booleans, bytes, objects, other sizes of floats or integers) are unsupported and files containing them will raise an error when loaded.

When serializing BQN arrays, the library uses the most restrictive dtype possible. An array consisting only of positive integers will be saved as <u4, otherwise <i4 if there are negative integers, otherwise <f8 for arbitrary numbers.

Usage 

⟨SaveNpy,LoadNpy⟩←•Import"npy.bqn"

# Deserialization
arr1←LoadNpy"arr1.npy"

# Serialization
arr2←2‿3‿4⥊•rand.Range 0  # will be saved as a floating point array
"arr2.npy" SaveNpy arr2

Tests

Run the Python script to generate test data: python gentest.py.

To run tests, run bqn tests.bqn.