Skip to main content

Command Palette

Search for a command to run...

Python API for the MediaPipe Object Detection Solution

Task 1 (GSoC 2021)

Published
2 min read

This is an extension to Mediapipe Object Detection Solution

Configuration Options

- max_object_detection
Maximum number of objects to detect. Default to 3.
- min_detection_confidence
Minimum confidence value ([0.0, 1.0]) from the object detection model for the detection to be considered successful. Default to 0.6
- min_suppression_threshold
Minimum suppression value ([0.0, 1.0]) from the object detection model for the detection to be considered successful. Default to 0.4

Python API

Please first follow general instructions to install the MediaPipe Python package, then learn more with the usage example below.

Supported configuration options:

max_object_detection
min_detection_confidence
min_suppression_threshold

max_object_detection=5 min_detection_confidence=0.6.gif

Usage

import cv2
import mediapipe as mp

mp_object_detection = mp.solutions.object_detection
mp_drawing = mp.solutions.drawing_utils

# For static images:
IMAGE_FILES = []
with mp_object_detection.ObjectDetection(
    min_detection_confidence=0.5) as object_detection:
  for idx, file in enumerate(IMAGE_FILES):
    image = cv2.imread(file)
    # Convert the BGR image to RGB and process it with MediaPipe Object Detection.
    results = object_detection.process(cv2.cvtColor(image, cv2.COLOR_BGR2RGB))

    # Draw the object detection annotations on the image
    # for each detected object.
    if not results.detections:
      continue
    annotated_image = image.copy()
    for detection in results.detections:
      mp_drawing.draw_detection(annotated_image, detection)
    cv2.imwrite('/tmp/annotated_image' + str(idx) + '.png', annotated_image)

# For webcam input:
cap = cv2.VideoCapture(0)
with mp_object_detection.ObjectDetection(
    max_object_detection=1,
    min_detection_confidence=0.6,
    min_suppression_threshold=0.4) as object_detection:
  while cap.isOpened():
    success, image = cap.read()
    if not success:
      print("Ignoring empty camera frame.")
      # If loading a video, use 'break' instead of 'continue'.
      continue

    # Flip the image horizontally for a later selfie-view display, and convert
    # the BGR image to RGB.
    image = cv2.cvtColor(cv2.flip(image, 1), cv2.COLOR_BGR2RGB)
    # To improve performance, optionally mark the image as not writeable to
    # pass by reference.
    image.flags.writeable = False
    results = object_detection.process(image)

    # Draw the object detection annotations on the image
    # for each detected object.
    image.flags.writeable = True
    image = cv2.cvtColor(image, cv2.COLOR_RGB2BGR)
    if results.detections:
      for detection in results.detections:
        mp_drawing.draw_detection(image, detection)
    cv2.imshow('MediaPipe Object Detection', image)
    if cv2.waitKey(5) & 0xFF == 27:
      break
cap.release()


GitHub

1.5K views
T
Thomas4y ago

Hi finally managed to compile the hello world example. Now compiled object_detection:object_detection_cpu. How do i run it?

I tried this:

C:\_Research\mediapipe-od-build\mediapipe\bazel-bin\mediapipe\examples\desktop\object_detection>object_detection_cpu.exe --calculator_graph_config_file=C:\_Research\mediapipe-od-build\mediapipe\mediapipe\modules\object_detection\object_detection_cpu.pbtxt
E20220124 16:58:32.190173  1836 demo_run_graph_main.cc:154] Failed to run the graph: ValidatedGraphConfig Initialization failed.
No registered object with name: DetectionLetterboxRemovalCalculator; Unable to find Calculator "DetectionLetterboxRemovalCalculator"
T
Thomas4y ago

I am using pycharm with the official mediapipe package and all solutions work just fine. is there a way to add your object detection part manually to the package?

I am not giving up yet:) but its pretty frustrating.

Also i can not find the binarypb file for object detection created anywhere. When building other examples like pose or hand there are also binarypb files created in bazel-out modules folder.

T
Thomas4y ago

I got rid of the initial errors but now stuck at some more: Any Idea? I do not even see what seems to be the problem:(

ERROR: C:/users/user/_bazel_user/o7dgz4zy/external/com_google_absl/absl/debugging/BUILD.bazel:31:11: Compiling absl/debugging/stacktrace.cc failed: (Exit 2): cl.exe failed: error executing command C:/Program Files (x86)/Microsoft Visual Studio/2019/BuildTools/VC/Tools/MSVC/14.29.30133/bin/HostX64/x64/cl.exe /nologo /DCOMPILER_MSVC /DNOMINMAX /D_WIN32_WINNT=0x0601 /D_CRT_SECURE_NO_DEPRECATE ... (remaining 41 argument(s) skipped)
A

It seems like a Windows problem with the C++ compiler and I don't have access to a Windows Machine currently. You may try opening an issue on Mediapipe Issue Tracker

T
Thomas4y ago

Aniket Ray yes. i have added like everything c++ related to vs build tools but seems thee is still something missing :( like: windows.h

T
Thomas4y ago

Is there a more detailed description of how to get this to run? I cloned your repo and followed Installing on Windows

On Step 9 the bazel build throws some errors: "GET returned 404 Not Found" So the files are not online anymore. I wonder if i am following the right path anyway.

Thanks in advance for any hints :)

c:\mediapipe-test\mediapipe>C:\Bazel\bazel.exe build -c opt --define MEDIAPIPE_DISABLE_GPU=1 --action_env PYTHON_BIN_PATH="C://python39//python.exe" mediapipe/examples/desktop/hello_world
Extracting Bazel installation...
Starting local Bazel server and connecting to it...
DEBUG: Rule 'rules_foreign_cc' indicated that a canonical reproducible form can be obtained by modifying arguments sha256 = "c2cdcf55ffaf49366725639e45dedd449b8c3fe22b54e31625eb80ce3a240f1e"
DEBUG: Repository rules_foreign_cc instantiated at:
  C:/mediapipe-test/mediapipe/WORKSPACE:42:13: in <toplevel>
Repository rule http_archive defined at:
  C:/users/user/_bazel_user/r4ook2nm/external/bazel_tools/tools/build_defs/repo/http.bzl:364:31: in <toplevel>
WARNING: Download from http://mirror.tensorflow.org/github.com/tensorflow/toolchains/archive/d781e89e2ee797ea7afd0c8391e761616fc5d50d.tar.gz failed: class java.io.FileNotFoundException GET returned 404 Not Found
WARNING: Download from http://mirror.tensorflow.org/github.com/bazelbuild/rules_closure/archive/cf1e44edb908e9616030cc83d085989b8e6cd6df.tar.gz failed: class java.io.FileNotFoundException GET returned 404 Not Found
WARNING: Download from http://mirror.tensorflow.org/github.com/tensorflow/runtime/archive/0a8c6996bd6a58eb0e7e2fe049ac67bf663e1950.tar.gz failed: class java.io.FileNotFoundException GET returned 404 Not Found
DEBUG: C:/users/user/_bazel_user/r4ook2nm/external/org_tensorflow/third_party/repo.bzl:109:14:
Warning: skipping import of repository 'com_google_absl' because it already exists.
DEBUG: C:/users/user/_bazel_user/r4ook2nm/external/org_tensorflow/third_party/repo.bzl:109:14:
Warning: skipping import of repository 'com_google_protobuf' because it already exists.
DEBUG: C:/users/user/_bazel_user/r4ook2nm/external/org_tensorflow/third_party/repo.bzl:109:14:
Warning: skipping import of repository 'com_google_googletest' because it already exists.
DEBUG: C:/users/user/_bazel_user/r4ook2nm/external/org_tensorflow/third_party/repo.bzl:109:14:
Warning: skipping import of repository 'com_github_gflags_gflags' because it already exists.
DEBUG: C:/users/user/_bazel_user/r4ook2nm/external/org_tensorflow/third_party/repo.bzl:109:14:
Warning: skipping import of repository 'build_bazel_rules_apple' because it already exists.
DEBUG: C:/users/user/_bazel_user/r4ook2nm/external/org_tensorflow/third_party/repo.bzl:109:14:
Warning: skipping import of repository 'build_bazel_rules_swift' because it already exists.
DEBUG: C:/users/user/_bazel_user/r4ook2nm/external/org_tensorflow/third_party/repo.bzl:109:14:
Warning: skipping import of repository 'build_bazel_apple_support' because it already exists.
DEBUG: C:/users/user/_bazel_user/r4ook2nm/external/org_tensorflow/third_party/repo.bzl:109:14:
Warning: skipping import of repository 'bazel_skylib' because it already exists.
DEBUG: C:/users/user/_bazel_user/r4ook2nm/external/org_tensorflow/third_party/repo.bzl:109:14:
Warning: skipping import of repository 'pybind11' because it already exists.
ERROR: C:/mediapipe-test/mediapipe/WORKSPACE:266:23: fetching android_ndk_repository rule //external:androidndk: Either the path attribute of android_ndk_repository or the ANDROID_NDK_HOME environment variable must be set.
ERROR: Analysis of target '//mediapipe/examples/desktop/hello_world:hello_world' failed; build aborted: Either the path attribute of android_ndk_repository or the ANDROID_NDK_HOME environment variable must be set.
INFO: Elapsed time: 42.124s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (7 packages loaded, 11 targets configured)
    Fetching @local_execution_config_python; fetching
A

Can you please try #70 and #5013 and let me know if this fixes your issue?

Also, make sure the ENV variables are correctly pointing to the locations.

1
M

i want object detection for my project but not working for me can u pls help .

mp_object_detection = mp.solutions.object_detection

its showing error here.

A

Hi! Can you please post the log output for the error?