Category : google-cloud-storage

Is this the most efficient way in python of finding blob storage object created between two dates? def objects_in_date_range(store, start_date, end_date): storage_client = storage.Client() bucket = storage_client.get_bucket(store) bucket_list = storage_client.list_blobs(bucket) for blob in bucket_list: name = blob.name created = blob.time_created # print(type(created), created) # print(type(start_date), start_date) if created >= start_date and created < end_date: yield ..

Read more

In local machine, I am able to extract the feature of images and store the corresponding .npy file in a different folder. from PIL import Image from feature_extractor import FeatureExtractor from pathlib import Path import numpy as np if __name__ == ‘__main__’: fe = FeatureExtractor() for img_path in sorted(Path(“./static/img”).glob(“*.jpg”)): print(img_path) # e.g., ./static/img/xxx.jpg feature = ..

Read more

I know how to download the file from cloud storage within the cloud run instance. But, I can’t find the syntax for reading the file in python. I’m looking to immediately convert the csv file into a pandas dataframe, just by using pd.read_csv(‘testing.csv’). So my personal code looks like, download_blob(bucket_name, source_blob_name, ‘testing.csv’). So shouldn’t I ..

Read more

I have been trying to create a GCP data bucket in my test gcp account assigned to me by my organisation. I keep getting a timeout error though and not sure why… any body able to help? code and error below. import os from google.cloud import storage os.environ["GOOGLE_APPLICATION_CREDENTIALS"]="testproject.json" storage_client = storage.Client() ## Create a bucket ..

Read more

Previously worked on .csv files which was straightforward to upload to GCS For csv I would do the following, which works: blob = bucket.blob(path) blob.upload_from_string(dataframe.to_csv(), ‘text/csv’) I am trying to do the same i.e. write the dataframe as a .feather file in bucket blob = bucket.blob(path) blob.upload_from_string(dataframe.reset_index().to_feather(), ‘text/feather’) However, this fails saying to_feather() requires a ..

Read more

I am using this piece of code to look at the response status: for i, status in enumerate(result.statuses): print(‘Status of processing line {} of the csv: {}’.format( i, status)) # Check the status of reference image # `0` is the code for OK in google.rpc.Code. if status.code == 0: reference_image = result.reference_images[i] print(reference_image) else: print(‘Status ..

Read more