Vision service API

Parameters:

• ctx (Context): A Context carries a deadline, a cancellation signal, and other values across API boundaries.
• cameraName (string): The name of the camera from which to get an image to run detections on.
• extra (map[string]interface{}): Extra options to pass to the underlying RPC call.

Returns:

• ([]objectdetection.Detection): A list of bounding boxes around the detected objects, and confidence scores of those detections.
• (error): An error, if one occurred.

Example:

myDetectorService, err := vision.FromProvider(machine, "my_detector")
if err != nil {
  logger.Error(err)
  return
}

// Get detections from the camera output
detections, err := myDetectorService.DetectionsFromCamera(context.Background(), "my_camera", nil)
if err != nil {
  logger.Fatalf("Could not get detections: %v", err)
}
if len(detections) > 0 {
  logger.Info(detections[0])
}

For more information, see the Go SDK Docs.

Parameters:

• cameraName (string) (required): The name of the camera to use for detection.
• extra (None) (optional)
• callOptions (CallOptions) (optional)

Returns:

• (Promise<visionApi.Detection[]>): * The list of Detections.

Example:

const vision = new VIAM.VisionClient(machine, 'my_vision');
const detections = await vision.getDetectionsFromCamera('my_camera');

For more information, see the TypeScript SDK Docs.

Parameters:

• cameraName String (required)
• extra Map<String, dynamic>? (optional)

Returns:

• Future<List<Detection>>

Example:

// Example:
var detections = await myVisionService.detectionsFromCamera('myWebcam');

For more information, see the Flutter SDK Docs.

Parameters:

• ctx (Context): A Context carries a deadline, a cancellation signal, and other values across API boundaries.
• img (image.Image): The image in which to look for detections.
• extra (map[string]interface{}): Extra options to pass to the underlying RPC call.

Returns:

• ([]objectdetection.Detection): A list of 2D bounding boxes around the detected objects, and confidence scores of those detections.
• (error): An error, if one occurred.

Example:

 // add "go.viam.com/rdk/utils" to imports to use this code snippet

  myCam, err := camera.FromProvider(machine, "my_camera")
  if err != nil {
    logger.Error(err)
    return
  }
  // Get an image from the camera decoded as an image.Image
  img, err = camera.DecodeImageFromCamera(context.Background(), utils.MimeTypeJPEG, nil, myCam)

  myDetectorService, err := vision.FromProvider(machine, "my_detector")
  if err != nil {
    logger.Error(err)
    return
  }
  // Get the detections from the image
  detections, err := myDetectorService.Detections(context.Background(), img, nil)
  if err != nil {
    logger.Fatalf("Could not get detections: %v", err)
  }
  if len(detections) > 0 {
    logger.Info(detections[0])
  }

For more information, see the Go SDK Docs.

Parameters:

• image (Uint8Array) (required): The image from which to get detections.
• width (number) (required): The width of the image.
• height (number) (required): The height of the image.
• mimeType (MimeType) (required): The MimeType of the image.
• extra (None) (optional)
• callOptions (CallOptions) (optional)

Returns:

• (Promise<visionApi.Detection[]>): * The list of Detections.

Example:

const camera = new VIAM.CameraClient(machine, 'my_camera');
const vision = new VIAM.VisionClient(machine, 'my_vision');

const { images } = await camera.getImages();
const detections = await vision.getDetections(
  images[0].image,
  600,
  600,
  images[0].mimeType
);

For more information, see the TypeScript SDK Docs.

Parameters:

• image ViamImage (required)
• extra Map<String, dynamic>? (optional)

Returns:

• Future<List<Detection>>

Example:

// Example:
var latestImage = await myWebcam.image();
var detections = await myVisionService.detections(latestImage);

For more information, see the Flutter SDK Docs.

Parameters:

• ctx (Context): A Context carries a deadline, a cancellation signal, and other values across API boundaries.
• cameraName (string): The name of the camera from which to get an image to run the classifier on.
• n (int): The number of classifications to return. For example, if you specify 3 you will get the top three classifications with the greatest confidence scores.
• extra (map[string]interface{}): Extra options to pass to the underlying RPC call.

Returns:

• (classification.Classifications): A list of classification labels, and the confidence scores of those classifications.
• (error): An error, if one occurred.

Example:

myClassifierService, err := vision.FromProvider(machine, "my_classifier")
if err != nil {
  logger.Error(err)
  return
}
// Get the 2 classifications with the highest confidence scores from the camera output
classifications, err := myClassifierService.ClassificationsFromCamera(context.Background(), "my_camera", 2, nil)
if err != nil {
  logger.Fatalf("Could not get classifications: %v", err)
}
if len(classifications) > 0 {
  logger.Info(classifications[0])
}

For more information, see the Go SDK Docs.

Parameters:

• cameraName (string) (required): The name of the camera to use for classification.
• count (number) (required): The number of Classifications requested.
• extra (None) (optional)
• callOptions (CallOptions) (optional)

Returns:

• (Promise<visionApi.Classification[]>): * The list of Classifications.

Example:

const vision = new VIAM.VisionClient(machine, 'my_vision');
const classifications = await vision.getClassificationsFromCamera(
  'my_camera',
  10
);

For more information, see the TypeScript SDK Docs.

Parameters:

• cameraName String (required)
• count int (required)
• extra Map<String, dynamic>? (optional)

Returns:

• Future<List<Classification>>

Example:

// Example:
var classifications = await myVisionService.classificationsFromCamera('myWebcam', 2);

For more information, see the Flutter SDK Docs.

Parameters:

• ctx (Context): A Context carries a deadline, a cancellation signal, and other values across API boundaries.
• img (image.Image): The image in which to look for classifications.
• n (int): The number of classifications to return. For example, if you specify 3 you will get the top three classifications with the greatest confidence scores.
• extra (map[string]interface{}): Extra options to pass to the underlying RPC call.

Returns:

• (classification.Classifications): A list of classification labels, and the confidence scores of those classifications.
• (error): An error, if one occurred.

Example:

 // add "go.viam.com/rdk/utils" to imports to use this code snippet

  myCam, err := camera.FromProvider(machine, "my_camera")
  if err != nil {
    logger.Error(err)
    return
  }
  // Get an image from the camera decoded as an image.Image
  img, err = camera.DecodeImageFromCamera(context.Background(), utils.MimeTypeJPEG, nil, myCam)

  myClassifierService, err := vision.FromProvider(machine, "my_classifier")
  if err != nil {
    logger.Error(err)
    return
  }
  // Get the 2 classifications with the highest confidence scores from the image
  classifications, err := myClassifierService.Classifications(context.Background(), img, 2, nil)
  if err != nil {
    logger.Fatalf("Could not get classifications: %v", err)
  }
  if len(classifications) > 0 {
    logger.Info(classifications[0])
  }

For more information, see the Go SDK Docs.

Parameters:

• image (Uint8Array) (required): The image from which to get classifications.
• width (number) (required): The width of the image.
• height (number) (required): The height of the image.
• mimeType (MimeType) (required): The MimeType of the image.
• count (number) (required): The number of Classifications requested.
• extra (None) (optional)
• callOptions (CallOptions) (optional)

Returns:

• (Promise<visionApi.Classification[]>): * The list of Classifications.

Example:

const camera = new VIAM.CameraClient(machine, 'my_camera');
const vision = new VIAM.VisionClient(machine, 'my_vision');

const { images } = await camera.getImages();
const classifications = await vision.getClassifications(
  images[0].image,
  600,
  600,
  images[0].mimeType,
  10
);

For more information, see the TypeScript SDK Docs.

Parameters:

• image ViamImage (required)
• count int (required)
• extra Map<String, dynamic>? (optional)

Returns:

• Future<List<Classification>>

Example:

// Example:
var latestImage = await myWebcam.image();
var classifications = await myVisionService.classifications(latestImage, 2);

For more information, see the Flutter SDK Docs.

Parameters:

• ctx (Context): A Context carries a deadline, a cancellation signal, and other values across API boundaries.
• cameraName (string): The name of the 3D camera from which to get point cloud data.
• extra (map[string]interface{}): Extra options to pass to the underlying RPC call.

Returns:

• ([]*viz.Object): A list of point clouds and associated metadata like the center coordinates of each point cloud.
• (error): An error, if one occurred.

Example:

mySegmenterService, err := vision.FromProvider(machine, "my_segmenter")
if err != nil {
  logger.Error(err)
  return
}
// Get the objects from the camera output
objects, err := mySegmenterService.GetObjectPointClouds(context.Background(), "my_camera", nil)
if err != nil {
  logger.Fatalf("Could not get point clouds: %v", err)
}
if len(objects) > 0 {
  logger.Info(objects[0])
}

For more information, see the Go SDK Docs.

Parameters:

• cameraName (string) (required): The name of the camera.
• extra (None) (optional)
• callOptions (CallOptions) (optional)

Returns:

• (Promise<commonApi.PointCloudObject[]>): * The list of PointCloudObjects.

Example:

const vision = new VIAM.VisionClient(machine, 'my_vision');
const pointCloudObjects =
  await vision.getObjectPointClouds('my_camera');

For more information, see the TypeScript SDK Docs.

Parameters:

• cameraName String (required)
• extra Map<String, dynamic>? (optional)

Returns:

• Future<List<PointCloudObject>>

Example:

// Example:
var ptCloud = await myVisionService.objectPointClouds('myCamera');

For more information, see the Flutter SDK Docs.

Parameters:

• ctx (Context): A Context carries a deadline, a cancellation signal, and other values across API boundaries.
• cameraName (string): The name of the camera to use for detection.
• opts (viscapture.CaptureOptions): Additional options to provide if desired.
• extra (map[string]interface{}): Extra options to pass to the underlying RPC call.

Returns:

• (viscapture.VisCapture): A class that stores all potential returns from the vision service. It can return the image from the camera along with its associated detections, classifications, and objects, as well as any extra info the model may provide.
• (error): An error, if one occurred.

Example:

// The data to capture and return from the camera
captOpts := viscapture.CaptureOptions{
  ReturnImage: true,
  ReturnDetections: true,
}
// Get the captured data for a camera
capture, err := visService.CaptureAllFromCamera(context.Background(), "my_camera", captOpts, nil)
if err != nil {
  logger.Fatalf("Could not get capture data from vision service: %v", err)
}
image := capture.Image
detections := capture.Detections
classifications := capture.Classifications
objects := capture.Objects

For more information, see the Go SDK Docs.

Parameters:

• cameraName (string) (required): The name of the camera to use for classification, detection, and segmentation.
• __namedParameters (CaptureAllOptions) (required)
• extra (None) (optional)
• callOptions (CallOptions) (optional)

Returns:

• (Promise< { classifications: visionApi.Classification[]; detections: visionApi.Detection[]; extra: undefined | Struct; image: undefined | Image; objectPointClouds: commonApi.PointCloudObject[]; }, >): * The requested image, classifications, detections, and 3d point cloud objects.

Example:

const vision = new VIAM.VisionClient(machine, 'my_vision');
const captureAll = await vision.captureAllFromCamera('my_camera', {
  returnImage: true,
  returnClassifications: true,
  returnDetections: true,
  returnObjectPointClouds: true,
});

For more information, see the TypeScript SDK Docs.

Parameters:

• ctx (Context): A Context carries a deadline, a cancellation signal, and other values across API boundaries.
• cmd (map[string]interface{}): The command to execute.

Returns:

• (map[string]interface{}): The command response.
• (error): An error, if one occurred.

Example:

myVisionSvc, err := vision.FromProvider(machine, "my_vision_svc")

command := map[string]interface{}{"cmd": "test", "data1": 500}
result, err := myVisionSvc.DoCommand(context.Background(), command)

For more information, see the Go SDK Docs.

Parameters:

• command (Struct) (required): The command to execute. Accepts either a Struct or a plain object, which will be converted automatically.
• callOptions (CallOptions) (optional)

Returns:

• (Promise<JsonValue>)

Example:

// Plain object (recommended)
const result = await resource.doCommand({
  myCommand: { key: 'value' },
});

// Struct (still supported)
import { Struct } from '@viamrobotics/sdk';

const result = await resource.doCommand(
  Struct.fromJson({ myCommand: { key: 'value' } })
);

For more information, see the TypeScript SDK Docs.

Parameters:

• command Map<String, dynamic> (required)

Returns:

• Future<Map<String, dynamic>>

Example:

// Example using doCommand with an arm component
const command = {'cmd': 'test', 'data1': 500};
var result = myArm.doCommand(command);

For more information, see the Flutter SDK Docs.

Parameters:

• None.

Returns:

• (Name)

Example:

myVisionSvc, err := vision.FromProvider(machine, "my_vision_svc")

err = myVisionSvc.Name()

For more information, see the Go SDK Docs.

Parameters:

• None.

Returns:

• (string): The name of the resource.

Example:

vision.name

For more information, see the TypeScript SDK Docs.

Parameters:

• name String (required)

Returns:

• ResourceName

Example:

final myVisionServiceResourceName = myVisionService.getResourceName("my_vision_service");

For more information, see the Flutter SDK Docs.

Parameters:

• ctx (Context): A Context carries a deadline, a cancellation signal, and other values across API boundaries.
• extra (map[string]interface{}): Extra options to pass to the underlying RPC call.

Returns:

• (*Properties)
• (error): An error, if one occurred.

For more information, see the Go SDK Docs.

Parameters:

• extra (None) (optional)
• callOptions (CallOptions) (optional)

Returns:

• (Promise< { classificationsSupported: boolean; detectionsSupported: boolean; objectPointCloudsSupported: boolean; }, >): * The properties of the vision service.

Example:

const vision = new VIAM.VisionClient(machine, 'my_vision');
const properties = await vision.getProperties();

For more information, see the TypeScript SDK Docs.

Parameters:

• extra Map<String, dynamic>? (optional)

Returns:

• Future<VisionProperties>

Example:

// Example:
var properties = await myVisionService.properties();
properties.detections_supported
properties.classifications_supported

For more information, see the Flutter SDK Docs.

Parameters:

• ctx (Context): A Context carries a deadline, a cancellation signal, and other values across API boundaries.

Returns:

• (error): An error, if one occurred.

Example:

myVisionSvc, err := vision.FromProvider(machine, "my_vision_svc")

err = myVisionSvc.Close(context.Background())

For more information, see the Go SDK Docs.