How to solve (pickle data was truncated) for file that stored large array?

I worked on a large number of images, where I extracted the image’s features using deep learning model then stored them in the array.
Because the process takes a long time, I saved this array in the Pickle file to use it again using the following code:

pickle.dump(array, open('Name-of-File.pickle', 'wb'))

When I want to load the data into an array from the Pickle file using the following code:

array = pickle.load(open('Name-of-File.pickle', 'rb'))

I get this error message:

UnpicklingError: pickle data was truncated

The file size is 1.94GB.

The environment machine is: Google Compute Engine Backend (GPU).

Available space: More than 100GB.

Any help without needing to reprocess the data again?
Thank you in advance.

Source: Python-3x Questions

LEAVE A COMMENT