I worked on a large number of images, where I extracted the image’s features using deep learning model then stored them in the array.
Because the process takes a long time, I saved this array in the Pickle file to use it again using the following code:
pickle.dump(array, open('Name-of-File.pickle', 'wb'))
When I want to load the data into an array from the Pickle file using the following code:
array = pickle.load(open('Name-of-File.pickle', 'rb'))
I get this error message:
UnpicklingError: pickle data was truncated
The file size is 1.94GB.
The environment machine is: Google Compute Engine Backend (GPU).
Available space: More than 100GB.
Any help without needing to reprocess the data again?
Thank you in advance.
Source: Python-3x Questions