AI Runtime Engineer - (Remote, US)

BEARCloud, Inc.

San Francisco, CA, USA

Published: 6/14/2022

Engineering

Full Time

Job Description

About the Opportunity

We are seeking an experienced AI Runtime Engineer to develop and optimize high-performance runtime software for advanced AI acceleration technologies.

In this role, you will focus on building low-latency, high-throughput execution systems that enable efficient deep learning workloads on specialized computing hardware. You will collaborate with hardware, compiler, and AI framework engineering teams to optimize AI inference and training performance across cloud and edge environments.

Key Responsibilities

Develop and optimize runtime software for executing deep learning workloads on AI accelerators and specialized hardware.
Implement efficient task scheduling, memory management, and kernel execution strategies.
Optimize data movement between host and device using PCIe, DMA, and shared memory.
Design and implement high-performance interfaces for AI inference frameworks, including ONNX Runtime, OpenVINO, and vLLM.
Improve graph execution performance through kernel fusion, pipelining, tensor tiling, and caching.
Integrate runtime components with compiler technologies such as LLVM, MLIR, XLA, and TVM.
Collaborate with cross-functional engineering teams to improve hardware utilization, execution efficiency, and overall system performance.
Support scalable and reliable AI runtime deployments across cloud and edge computing environments.

Required Qualifications

Bachelor's or Master's degree in Computer Science, Electrical Engineering, or a related technical discipline.
3+ years of experience developing low-level runtime software for AI accelerators, GPUs, or high-performance computing (HPC) systems.
Strong proficiency in C/C++ and low-level systems programming.
Solid understanding of task scheduling, concurrency, memory hierarchy, and resource management.
Experience with hardware-aware performance optimization and dataflow architectures.
Familiarity with deep learning execution frameworks such as ONNX Runtime, TensorRT, TVM, or OpenVINO.
Experience optimizing low-latency, high-throughput AI workloads.
Strong debugging, profiling, and performance analysis skills.

Preferred Qualifications

Experience with AI compiler technologies, including LLVM, MLIR, XLA, or TVM.
Knowledge of PCIe, DMA, memory optimization, and hardware/software integration.
Exposure to AI model deployment technologies such as NVIDIA Triton Inference Server or TensorFlow Serving.
Experience supporting AI inference workloads across heterogeneous computing environments.

Why This Opportunity?

This position offers the opportunity to work on advanced AI computing technologies, solve complex systems-level performance challenges, and contribute to the development of high-performance AI infrastructure.