Tensorflow – Showing sorted predictions from left to right by tracking bounding boxes and outputting them

I’m trying to make a prediction with tf 2.0, I managed to train my model and to show the output on the image by printing the bounding boxes on it, but I’m struggling to output the predicitons as output in the a "left to right" order.

I thought it would be easier to work on the bounding boxes, so I put the xmin coords in a numpy array and was trying to match the xmin_arr contents with the box[0] (in the for:

i=0
  for box in b:
    print ("This box with xmin", box[0], "is gonna get used, Detected class:", category_index[ output_dict['detection_classes'][i]])
    i+=1

) but I don’t think that’s the best way and also it is not correct.

This is what I’ve done so far:

def show_inference_and_prediction(model, image_np):
  # printing img height and width, I don't use it in this function
  # but I used it before to get xmin coords.
  height, width, _ = img.shape
  print ("IMG Height:", height, "IMG Width", width)
   # Actual detection.
  output_dict = run_inference_for_single_image(model, image_np)

  #get the detected class sorted by detection scores
  indexes = [i for i,k in enumerate(output_dict['detection_scores']) if (k > 0.8)]
  class_id = itemgetter(*indexes)(output_dict['detection_classes'])
  class_names = []
  for i in range(0, len(indexes)):
    class_names.append(category_index[class_id[i]]['name'])
  print("Detected classes:", class_names,"nn")
 
 
  boxes = output_dict['detection_boxes']
  # get all boxes from an array
  max_boxes_to_draw = boxes.shape[0]
  # get scores to get a threshold
  scores = output_dict['detection_scores']
  # threshold
  min_score_thresh=0.8
  xmin_arr=[]
  
  # iterate over all objects found
  for i in range(min(max_boxes_to_draw, boxes.shape[0])):
    # 
    if scores is None or scores[i] > min_score_thresh:
        xmin = output_dict['detection_boxes'][i][0]
        class_name = category_index[output_dict['detection_classes'][i]]['name']
        print ("This box is gonna get used", boxes[i][0], output_dict['detection_classes'][i])
        
        #print(ymin, xmin, ymax, xmax)
        xmin_arr.append(xmin)
      
  print("Non sorted xmin_arr",xmin_arr)
  xmin_arr.sort()
  print("Sorted xmin_arr", xmin_arr, "nn")

  boxes_test = np.squeeze(output_dict['detection_boxes'])
  scores_test = np.squeeze(output_dict['detection_scores'])
  bboxes = boxes[scores_test > min_score_thresh]
  print("Non sorted numpy array")
  print(bboxes,"nn")
  


  print("Sorted numpy array by xmin")
  ind=np.argsort(bboxes[:,0])
  b=bboxes[ind]
  print(b,"nn")
  #I know this isn't the best way for a for loop... i'm just new to python
  i=0
  for box in b:
    print ("This box with xmin", box[0], "is gonna get used, Detected class:", category_index[ output_dict['detection_classes'][i]])
    i+=1
  # Visualization of the results of a detection.
  final_img =vis_util.visualize_boxes_and_labels_on_image_array(
          image_np,
          output_dict['detection_boxes'],
          output_dict['detection_classes'],
          output_dict['detection_scores'],
          category_index,
          instance_masks=output_dict.get('detection_masks_reframed', None),
          use_normalized_coordinates=True,
          line_thickness=6)
  return(final_img)

and this is the output

IMG Height: 100 IMG Width 220
Detected classes: ['6', '0', '6', '5', '0', '+'] 


This box is gonna get used 0.15368861 6
This box is gonna get used 0.25094065 10
This box is gonna get used 0.5650149 6
This box is gonna get used 0.53073287 5
This box is gonna get used 0.21016338 10
This box is gonna get used 0.48348305 11
Non sorted xmin_arr [0.15368861, 0.25094065, 0.5650149, 0.53073287, 0.21016338, 0.48348305]
Sorted xmin_arr [0.15368861, 0.21016338, 0.25094065, 0.48348305, 0.53073287, 0.5650149] 


Non sorted numpy array
[[0.15368861 0.00103605 0.4914853  0.14996211]
 [0.25094065 0.24868643 0.6210675  0.4069612 ]
 [0.5650149  0.81631124 0.9563305  0.9875988 ]
 [0.53073287 0.6841933  0.9102581  0.82026345]
 [0.21016338 0.1524337  0.5577521  0.27355438]
 [0.48348305 0.46985003 0.7388715  0.5943037 ]] 


Sorted numpy array by xmin
[[0.15368861 0.00103605 0.4914853  0.14996211]
 [0.21016338 0.1524337  0.5577521  0.27355438]
 [0.25094065 0.24868643 0.6210675  0.4069612 ]
 [0.48348305 0.46985003 0.7388715  0.5943037 ]
 [0.53073287 0.6841933  0.9102581  0.82026345]
 [0.5650149  0.81631124 0.9563305  0.9875988 ]] 


This box with xmin 0.15368861 is gonna get used, Detected class: {'id': 6, 'name': '6'}
This box with xmin 0.21016338 is gonna get used, Detected class: {'id': 10, 'name': '0'}
This box with xmin 0.25094065 is gonna get used, Detected class: {'id': 6, 'name': '6'}
This box with xmin 0.48348305 is gonna get used, Detected class: {'id': 5, 'name': '5'}
This box with xmin 0.53073287 is gonna get used, Detected class: {'id': 10, 'name': '0'}
This box with xmin 0.5650149 is gonna get used, Detected class: {'id': 11, 'name': '+'}

The thing is: the input image shows: 606+56.
And the prediction on the classes are correct, they’re just not sorted.
I think the error is at:

print ("This box with xmin", box[0], "is gonna get used, Detected class:", category_index[ output_dict['detection_classes'][i]])

Because it just works on the old indexes…

I would be glad if someone could help me to sort this out.
Thank you in advance.

Source: Python Questions

LEAVE A COMMENT