AirGen Utilities

general

airgen.utils.general.get_public_fields(obj)

get all public fields of an object (not starting with _)

Parameters:

obj (Object)

Return type:

List[str]

airgen.utils.general.list_to_2d_float_array(flst, width, height)

convert a list of floats to a 2d array

Parameters:

  • flst (List[float])

  • width (int)

  • height (int)

Returns:

numpy array with dtype=np.float32, shape=(height, width)

Return type:

np.ndarray

airgen.utils.general.string_to_float_array(bstr)

convert a string to a float array

Parameters:

bstr (str|bytes) -- _description_

Return type:

np.ndarray

airgen.utils.general.string_to_uint8_array(bstr)

convert a string to a uint8 array

Parameters:

bstr (str|bytes)

Returns:

numpy array with dtype uint8

Return type:

np.ndarray

airgen.utils.general.to_dict(obj)

Convert an object to a dictionary of its public fields

Parameters:

obj (Object)

Return type:

dic (Dict[str, Any])

airgen.utils.general.to_str(obj)

Convert an object to a string of its public fields

Parameters:

obj (object)

Return type:

str

airgen.utils.general.vector3d2list(vector3d)

turn an AirGen Vector3r into a list of three floats

Parameters:

vector3d (Vector3r)

Return type:

List[float]

airgen.utils.general.wait_key(message='')

Wait for a key press on the console and return it.

airgen.utils.general.write_file(filename, bstr)

Write binary data to file. Used for writing compressed PNG images

sensor

airgen.utils.sensor.imagetype2request(camera_name, image_type)

helper function that creates uncompressed AirGen ImageRequest for image type

Parameters:

  • camera_name (Literal["front_center", "bottom_center"])

  • image_type (ImageType)

Return type:

ImageRequest

airgen.utils.sensor.responses2images(responses)

help function that converts a list of AirGen ImageResponse to a list of (image, camera_params) tuple

Parameters:

responses (List[ImageResponse]) -- list of AirGen ImageResponse

Raises:

ValueError -- if response' image type is among listed below

Returns:

list of (image, camera_params) tuple

Return type:

List[Tuple[np.ndarray, Dict[str, Any]]]

image:

  • For ImageType.Scene, ImageType.DepthVis, ImageType.OpticalFlowVis, image data is RGB image, numpy array with dtype=np.uint8, shape=(height, width, 3)

  • For ImageType.DepthPerspective, ImageType.DepthPlanar, ImageType.DisparityNormalized, image data is depth image in meters, numpy array with dtype=np.float32, shape=(height, width, 1)

  • For ImageType.Segmentation, image data is array of objects' SegmentationIDs, numpy array with dtype=np.uint8, shape=(height, width, 1)

  • For ImageType.SurfaceNormals, image data is array of unit vector (normalized length), numpy array with dtype=np.float32, shape=(height, width, 3)

  • For ImageType.OpticalFlow, image data is array of displacements in x,y directions, numpy array with dtype=np.float32, shape=(height, width, 2)

  • For ImageType.Infrared, image data is array of infrared values (object's SegmentationID for now), numpy array with dtype=np.uint8, shape=(height, width, 1)

camera parameters:

  • width (int): image width

  • height (int): image height

  • fov (float): camera field of view in degrees

  • camera_position (List[float]): camera position (x,y,z) in airgen (NED frame)

  • camera_orientation_euler_pry (List[float]): camera orientation (pitch, roll, yaw) in degrees

  • camera_orientation_quat_wxyz (List[float]): camera orientation (w,x,y,z) in quaternion format

mechanics

airgen.utils.mechanics.cameracoord2worldcoord(camera_coord, camera_params)

given airgen camera parameters, transform camera coordinate back to airgen world coordinate

Parameters:

  • camera_coord (List[float]) -- (x,y,z)

  • camera_params (dict) -- camera parameters

Returns:

_description_

Return type:

np.ndarray

airgen.utils.mechanics.homo_coord_to_nonhome_coord(home_coord)

turn homogeneous coordinates to non-homogeneous coordinates (factoring out the last dimension)

Parameters:

home_coord (ivy.Array) -- of shape (..., n)

Returns:

of shape (..., n-1)

Return type:

ivy.Array

airgen.utils.mechanics.imagecoord2direction(pixelcoord, camera_param)

Given camera parameters (position, pose, and fov) and pixel coordinate, return the 3D direction of the pixel with respect to the camera

Parameters:

  • pixelcoord (Tuple[float, float]) -- coordinate of the pixel in the image in xy format

  • camera_param (Dict[str, Any]) -- camera parameters

Returns:

normalized unit (directional) vector (x, y, z)

Return type:

Tuple[float, float, float]

airgen.utils.mechanics.imagecoord2orientation(pixelcoord, camera_param)

Given camera parameters (position, pose, and fov) and pixel coordinate, return the 3D direction of the pixel with respect to the camera represented in yaw and pitch (absolute degrees)

Parameters:

  • pixelcoord (Tuple[float, float]) -- coordinate of the pixel in the image in xy format

  • camera_param (Dict[str, Any]) -- camera parameters

Returns:

target pitch, roll, yaw in radians

Return type:

Tuple[float, float float]

airgen.utils.mechanics.imagecoord2pose(pixelcoord, point_depth, camera_param)

convert pixel coordinate to 3D coordinate

Parameters:

  • pixelcoord (Tuple[float, float]) -- coordinate of the pixel in the image in xy format

  • point_depth (float) -- depth of the point

  • camera_param (Dict[str, Any]) -- camera parameters

Returns:

target coordinate in (x, y, z) Tuple[float, float, float]: target orientation in (pitch, roll, yaw) in radians

Return type:

Tuple[float, float, float]

airgen.utils.mechanics.pose2vector(pitch, roll, yaw)

transform target direction represnted in (pitch, roll, yaw) in radians to directional vector

Parameters:

  • pitch (float) -- in radians

  • roll (float) -- in radians

  • yaw (float) -- in radians

Returns:

unit directional vector (x,y,z)

Return type:

Tuple[float, float, float]

airgen.utils.mechanics.quat_wxyz_to_xyzw(quat_wxyz)

transform quaternion (represented by list of floats) from wxyz format to xyzw format

Parameters:

quat_wxyz (List[Float])

Returns:

quaternion in xyzw format

Return type:

np.ndarray

airgen.utils.mechanics.rotate_xy(vec, theta)

rotate xy-component of 3d vector by theta (in degrees) counter-clockwise (in xy plane)

Assume looking from positive z-axis, the orientation is

^ y
|
|
|______> x
Parameters:

  • vec (np.ndarray) -- shape (3,)

  • theta (float) -- angles in degrees

Returns:

rotated vector shape (3,)

Return type:

np.ndarray

airgen.utils.mechanics.to_eularian_angles(q)

transform from quaternion to euler angles

Parameters:

q (Quaternionr) -- quaternion in wxyz format

Returns:

pitch, roll, yaw in radians

Return type:

Tuple[float, float, float]

airgen.utils.mechanics.vec2eulerangles(vec)

transform airgen directional vector to euler angles

Parameters:

vec (ndarray) -- directional vector of shape (N, 3)

Returns:

euler angles of shape (N, 3), each row is (pitch, roll, yaw) in degrees

Return type:

np.ndarray

vision

airgen.utils.vision.build_camera(camera_params)

given airgen camera parameters, build camera inverse extrinsic matrix and camera intrinsic matrix

Parameters:

camera_params (dict) -- airgen camera parameters

Returns:

inverse of camera extrinsic matrix and inverse of camera calibration matrix

Return type:

Tuple[ivy.Array, ivy.Array]

airgen.utils.vision.build_camera_intrinsic(camera_params)

given airgen camera parameters, build camera intrinsic matrix

Parameters:

camera_params (dict) -- aigen camera parameters

Return type:

ivy_vision.Intrinsics

airgen.utils.vision.build_camera_inv_extrinsic(camera_params)

given airgen camera parameters, build camera inverse extrinsic matrix

Parameters:

camera_params (dict) -- airgen camera parameters

Returns:

inverse of camera extrinsic matrix

Return type:

ivy.Array

airgen.utils.vision.camera_unproject_depth(depth, cam_inv_ext_mat, cam_inv_calib_mat)

generate point cloud from depth image (depth perspective)

Parameters:

  • depth (np.ndarray) -- of shape (H, W, 1)

  • cam_inv_ext_mat (ivy.Array) -- inverse of camera extrinsic matrix

  • cam_inv_calib_mat (ivy.Array) -- inverse of camera intrinsic matrix

Returns:

point cloud of shape (N, 3)

Return type:

np.ndarray

airgen.utils.vision.depth2pointcloud(depth, camera_param, mask=None)

generating point cloud from depth image

Parameters:

  • depth (np.ndarray) -- depth image of shape (H, W, 1)

  • camera_param (dict) -- camera parameters that contains fov, height, width and camera pose

  • mask (Optional[np.ndarray], optional) -- boolean (0/1) mask where 1 indicates object of interest, (H, W, 1). Defaults to None.

Returns:

point cloud in airgen world coordinate of shape (N, 3)

Return type:

np.ndarray

trajectory

class airgen.utils.trajectory.Waypoint(x, y, z, yaw, time)

Store a waypoint.

x,y,z

position in world frame

yaw

Euler angle of waypoint

time

time at which the waypoint is to be reached

airgen.utils.trajectory.calc_time(start, end)

Calculate a fitting time for the trajectory between two waypoints.

airgen.utils.trajectory.fit_min_snap_trajectory(waypoint_arr, isJoint=True)

Starting point of any generation.

Waypoints are given as an array of arrays and transformed into an array of Waypoints.

airgen.utils.trajectory.joint(waypoints)

Calculate a trajectory by a joint operation.

airgen.utils.trajectory.separate(waypoints)

Calculate a trajectory by separate operations.

visualize

airgen.utils.visualize.rr_camera_capture_logger(rr_space_view_name)

Given a view space name (entity_path in rerun), return a function that can be used to log camera data (pose and capture in AirGen's 3D world coordinate) to that view space.

Parameters:

  • rr_view_space_name (str) -- name of the view space (entity_path) in rerun

  • rr_space_view_name (str)

Returns:

a function that can be used to log camera data to that view space.

def log_camera_data(image: np.ndarray, camera_params: Dict[str, Any], camera_name: str):

pass

Return type:

Callable

Usage:
>>> trajectory_logger = rr_camera_capture_logger("airgen_world")
>>> trajectory_logger(image, camera_params, camera_name)
airgen.utils.visualize.rr_log_airgen_image(entity_path, image_type, image, image_name='')

log images from airgen to rerun

Parameters:

  • entity_path (str) -- _description_

  • image_type (ImageType) -- _description_

  • image (np.ndarray) -- _description_

  • image_name (str, optional) -- _description_

Raises:

ValueError -- ValueError if image_type is not supported