I cache a function which is called with large numpy arrays. There is an efficient way to hash them with https://github.com/ifduyue/python-xxhash
I think it's a common use case, so I can merge my method to the default hasher in this library.
The only problem is that hash collisions are possible, and the cache would then work incorrectly, but it's very rare, and also the existing functionality isn't broken. I can also add an option to not hash them but just convert to string with array.tobytes(), which prevents collisions but is only is suited for small arrays.
The same can be done for regular python lists (with xxhash or str(list)).
If you want, I can make a PR with this.
I cache a function which is called with large numpy arrays. There is an efficient way to hash them with https://github.com/ifduyue/python-xxhash
I think it's a common use case, so I can merge my method to the default hasher in this library.
The only problem is that hash collisions are possible, and the cache would then work incorrectly, but it's very rare, and also the existing functionality isn't broken. I can also add an option to not hash them but just convert to string with
array.tobytes(), which prevents collisions but is only is suited for small arrays.The same can be done for regular python lists (with xxhash or
str(list)).If you want, I can make a PR with this.