• K1是一颗8核64位RISC-V AI CPU
• 以RISC-V开源指令集为基础，致力打造更高能效、更通用的AI处理器平台，推动全球开源、开放的算力生态建设。
• K1主要应用在单板计算机（SBC）、智能机器人、运行端侧大模型的各种终端、家庭存储与计算终端、行业计算终端、AI PC、边缘节点计算机等领域。

联系我们: business@spacemit.com

开发者社区

合作伙伴开发板

• 首页
• 文档
• 活动
• 课程
• SDK
• Bianbu
• 论坛
• 云设备
• 工单
• 中文
• 登录

让RISC-V开发更简单

基于进迭时空的处理器和软件解决方案,可以快速开发各种产品

基于Bianbu操作系统和生态产品,可以轻松实现应用开发

Bianbu

针对RISC-V架构的处理器做了深度优化的操作系统 Desktop/Nas等版本,适用于不同产品的领域

MUSE Book

MUSE Box

MUSE Shelf

MUSE Pi

MUSE Card

MUSE N1

• 首页
• 文档
• 活动
• 课程
• SDK
• Bianbu
• 论坛
• 云设备
• 工单
• 中文
• 登录

首页

Key Stone

Power Stone

MUSE

B1

Bianbu

AI

Bianbu Cloud

OpenHarmony

Tools

Partners

FAQ

比赛专区

Key Stone

以RISC-V开源指令集为基础，致力打造更高能效、更通用的AI处理器平台，推动全球开源、开放的算力生态建设。

Power Stone

为高性能计算系统打造高效、稳定、灵活的定制化电源解决方案。

MUSE

构建完善易用的生态产品序列，打造RISC-V硬件产品标杆，为开发者提供更便捷的开发服务。

Bianbu

Bianbu是一个操作系统，针对RISC-V架构的处理器做了深度优化，有Bianbu Desktop NAS等版本，适用于不同的产品领域。

AI

以高性能CPU为基础，融合原生AI算力，提供完备开发套件，降低AI开发者门槛。

Tools

进迭时空为提高开发者效率开发的各种工具，包括AI部署工具、刷机工具和写号工具等。

赛事专区

高校赛事产品包资料专区，为参赛学子提供探索RISC-V与AI交叉领域的开源开放优质平台

Accept《SpacemiT Privacy Policy》

Register >

简介

Bianbu Linux 是 Spacemit Stone 系列芯片的 BSP，即 SDK。包含监管程序接口（OpenSBI）、引导加载程序（U-Boot/UEFI）、Linux 内核、根文件系统（包含各种中间件和库）以及示例等。其目标是为客户提供处理器 Linux 支持，并且可以开发驱动或应用。

系统架构​

主要组件​

以下是Bianbu Linux的组件：

• OpenSBI
• U-Boot
• Linux Kernel
• Buildroot
• onnxruntime (with Hardware Accelerated)
• ai-support: AI demo
• img-gpu-powervr: GPU DDK
• mesa3d
• QT 5.15 (with GPU enabled)
• k1x-vpu-firmware: Video Process Unit firmware
• k1x-vpu-test: Video Process Unit test program
• k1x-jpu: JPEG Process Unit API
• k1x-cam: CMOS Sensor and ISP API
• mpp: Media Process Platform
• FFmpeg (with Hardware Accelerated)
• GStreamer (with Hardware Accelerated)
• v2d-test: 2D Unit test program
• factorytest: factory test app

更多组件正在适配中。

快速指南​

• 镜像
• 源码
• 工具

更新说明​

• 更新说明

Skip to main content

All outgoing email has been globally disabled by an administrator. No email notifications of any kind will be sent.

1. categories
2. tags

• Latest
• Hot
• Categories

Topic list, column headers with buttons are sortable. Topic Posters Muse Pi Pro通过SD卡启动失败

MUSE Pi

求助答疑

求助：系统装在SD卡里，用的是openEuler2403sp1，然后有一天断电关机了，重启之后，板子灯正常亮，但是系统没有启动。 连上串口，看到打印 SD卡换到另一块板子上是可以正常启动的，是不是SD卡模块出问题了，该怎么解决啊？… read more 6 3h K1 IME拓展 矩阵乘 程序执行错误

K1

我使用 vmdot 指定实现了一个简单的矩阵乘算法，验证功能。在运行时出现了 segmentfault 问题，求大佬解答。以及是否有编程手册及程序示例 #include <stdio.h> #include <stdint.h> int… read more 51 1d
$bug$
SpaceMITExecutionProvider推理bug

K1

求助答疑

部分量化后模型使用providers=
$“SpaceMITExecutionProvider”$
出现识别与boxes与scores错误。已反馈问题。 13 1d vmadot的使用

K1

求助答疑

BianbuCloud上面ISA中没有IME扩展，如何使用vmadot？直接使用vmadot，加-mcpu=spacemit-x60编译，运行时还是会出现“非法指令（核心已转储）”。 19 6d 在Bit-Brick Cluster K1使用distcc集群编译

RISC-V

操作系统应用开发

在Bit-Brick Cluster K1使用distcc集群编译 基础环境 Bit-Brick Cluster K1 千兆网线 Bit-Brick Cluster K1 是一款高性能计算扩展设备，通过集成多个核心板构建计算… read more 57 7d 请问bianbu linux已经支持了哪些摄像头?

Bianbu OS

摄像头

如题. 没有找到这方面的信息, 所以想知道哪些摄像头已经有了官方支持. OV5647有吗? 23 9d MUSE Pi识别不到插入的SSD

MUSE Pi

38 9d 进迭时空双周报（20250704-0718）

社区动态

求助答疑公告

一.开发者常见问题 1.已解决问题 问题类型 链接 工具链与开发环境 工具链与开发环境 内核与底层调试 内核与底层调试 系统镜像与部署 系统镜像与部署 工具链与开发环境 【问题】K1能连vscode吗 【回… read more 25 9d There is NO dtb of `SpacemiT` in Debian riscv64 ISO.

K1

操作系统

Hey! SpacemiT, I noticed that Debian has added riscv64 as a release architecture ( Bits from the Release Team: hard fro… read more 121 11d
$felix86$
Mesa: Add gpu drivers in default Bianbu OS build

Bianbu OS

We are testing graphics x86_64 applications in Bianbu OS so we need proper graphics drivers for it: llvmpipe: it’s li… read more 44 13d Godot引擎无法使用vulkan图形后端

Bianbu OS

求助答疑gpu

目前gles后端是好用的，但是vulkan后端无法正常运行 新建一个Forward+项目就可以复现了 Vulkan 1.3.277 - Forward+ - Using Device #0: ImgTec - PowerVR B-… read more 100 13d 【UEFI 开源系列第三篇】基于MUSE Pi Pro的uefi固件制作

社区动态

在此系列前两篇文章中，我们介绍了 UEFI 的基础概念与核心组件，以及如何将 UEFI 适配到 RISC-V 架构的 SpacemiT K1 上。作为系列的第三篇内容，本文将以 MUSE Pi Pro 开发板为例，完整展示如何基于 S… read more 65 15d 【UEFI 开源系列第二篇】UEFI 在 SpacemiT K1 上的解决方案

社区动态

UEFI on Spacemit K1 进迭开源uefi源码仓库： https://gitee.com/bianbu-linux/edk2-platforms https://gitee.com/bianbu-linux/edk2 … read more 28 15d 其他板卡何时支持UEFI启动和配置

工具

看到新上架的MUSE Pi Pro支持UEFI启动和配置，其他MUSI板卡包括香蕉派这样的第三方板卡后续会支持UEFI固件吗，有无大致计划？ 107 15d 解析RISCV fence指令

RISC-V

背景 由于RISCV采用了弱内存模型RVWMO，对内存访问的约束较少，而在现代的处理器设计中，因为流水线，乱序执行，store buffer等的存在，对IO, memory的访问往往是乱序的。 而软件在特定的场景下，需要硬件对IO，mem… read more 534 16d 【UEFI 开源系列第一篇】什么是 UEFI？

社区动态

进迭开源uefi代码仓库： Bianbu Linux/edk2-platforms 【UEFI 开源系列第一篇】什么是 UEFI？ UEFI是什么？ UEFI（Unified Extensible Firmware Interfac… read more 31 17d 是否会提供 ARM 版本的交叉编译工具链

工具

https://archive.spacemit.com/toolchain/ 目前只有 x86-64 的交叉编译工具链,请问未来是否会提供 arm64 版本的工具链 37 19d 配置使用摄像头连接GMSL（如max9295+max9296），如何修改代码？

K1

求助答疑多媒体摄像头驱动

请问在k1板子上，想要使用imx415配置使用摄像头连接GMSL（如max9295+max9296），如何修改代码才能实现？ 31 22d musebook开不了机了

MUSE Book

描述： 运行完程序后合上笔记本，下次开机的时候就打不开了，长按电源键指示灯熄灭后重新按电源键启动，之后指示灯会亮并出现开机的界面，但是图标闪过后屏幕就变黑了，然后按什么都没反应，指示灯还在亮 38 23d 进迭时空双周报（20250619-0703）

社区动态

求助答疑公告

一.开发者常见问题 1.已解决问题 问题类型 链接 多媒体处理与编码 多媒体处理与编码 工具链与开发编译 工具链与开发编译 系统与内核 系统与内核 硬件与调试 硬件与调试 多媒体处理与编码 【问题】如… read more 32 23d 甜“芯”代码局沈阳站活动圆满成功！

社区动态

精彩活动

甜“芯”代码局：RISC-V 与 openKylin 的开源狂欢派对！圆满成功！ 感谢进迭时空对本场活动的提供RISC-V板卡、笔记本电脑支持！！！ 22 26d

K1系列芯片：8核 64 位 RISC-V AI CPU

K1

K1系列芯片 是进迭时空自研8核64位RISC-V AI CPU，以RISC-V开源指令集为基础，致力打造更通用、更高能效的AI处理器平台，推动全球开源、开放的AI算力生态建设。 K1系列AI CPU主要应用在单板计算机、网络存储、云电脑… read more 619 27d 甜“芯”代码局：RISC-V 与 openKylin 的开源狂欢派对！欢迎参加！

社区动态

操作系统新闻资讯精彩活动

活动名称：甜“芯”代码局：RISC-V 与 openKylin 的开源狂欢派对 活动时间：2025-07-03 14:00 至 2025-07-03 18:00 活动地点：沈阳市大东区中街吾悦广场tea’times茶时代 活动介… read more 28 Jul 1 x100 core未来会有开发板吗？

其他

我想要一款高性能的RV开发板，但不想要服务器形态的机器，问一下x100 core未来会出开发板吗或者PC机吗？ 156 Jun 30 如何修改 Muse Pi Pro SPI 上的 u-boot 和 UEFI 固件

MUSE Pi

求助答疑启动

我想通过 u-boot 或者 UEFI 进行 PXE 启动， 但是目前 Muse Pi Pro 的 UEFI 并不包含适配 Snp 协议的网卡驱动，所以无法联网， 而 u-boot 可以进行 PXE 启动，却因为其启动参数默认为 boo… read more 53 Jun 29 有没有最新版工具链?

工具

板子上自带的工具链不支持rvv intrinsic v1.0, 进迭时空没有再更新过工具链吗? 34 Jun 26 【进迭时空双周报】-（20250604-0618）

社区动态

求助答疑公告

一.开发者常见问题 1.已解决问题 问题类型 链接 工具安装与使用 工具安装与使用 硬件功能与性能 硬件功能与性能 驱动与系统兼容性 驱动与系统兼容性 产品相关 产品相关 工具安装与使用 【问题】这个… read more 50 Jun 20 perf record捕获不到数据

Bianbu OS

[# sudo perf record -F 999 ./sample 2000

$perfrecord:Wokenup1timestowritedata$
[ perf record: Captured and wro… read more](https://forum.spacemit.com/t/topic/323) 263 [Jun 19](https://forum.spacemit.com/t/topic/323/ ...

 截至2024年12月，进迭时空8核RISC-V AI CPU芯片K1累计量产交付量已突破5万颗，是量产速度最快的RISC-V高算力芯片，目前在工业、电力、机器人、运营商、消费电子等多个行业完成量产应用。

截至2024年12月，进迭时空8核RISC-V AI CPU芯片K1累计量产交付量已突破5万颗，是量产速度最快的RISC-V高算力芯片，目前在工业、电力、机器人、运营商、消费电子等多个行业完成量产应用。

开发者

单板计算机（SBC）

各生态伙伴基于K1芯片打造的的单板计算机（SBC），凭借RISC-V AI CPU的创新架构及其2TOPS融合AI算力，可快速接入Deepseek、通义千问等主流AI大模型生态；基于其丰富的 PCIe、USB、GMAC、CAN-FD等高速接口，以及工业级标准品质，可直接应用于网络存储、工业控制、边缘计算、智能机器人、云电脑等领域。

智算主控

工业PLC

基于K1芯片打造的可编程控制器（PLC），依托K1的高性能算力以及多路CAN-FD、千兆网口等丰富工业接口，可配置支持EtherCAT、Profinet等多种主站，符合IEC61499标准并配套编程软件免费使用，可广泛应用于各类自动化系统应用场景。

电力融合终端

搭载K1芯片的电力融合终端，内置 DeepSeek 本地大模型，实现全设备自然语言交互；无需协议适配，实时语音查询设备状态、能耗分析、故障预警，保障配电房安全高效运行；本地化部署确保数据隐私，8核算力支撑毫秒级响应，是智能配电房的全能管家。

AI NAS

基于K1芯片打造的AI NAS，小巧精致可随身携带，专为创意和专业人士设计，可以在不需要互联网连接的情况下进行即时、安全的文件传输，并支持Wi-Fi 6、数据同步和高速远程访问。内置电池备份、移动电源兼容、AI数据管理，非常适合在旅途中管理大量数据。

边缘计算网关

搭载K1芯片的的边缘计算网关，最高配置2路千兆网口+2路2.5G网口，可支持各类分布式网络协议栈流畅运行，内置2.0Tops融合AI算力，可部署DeepSeek等本地大模型，广泛应用于边缘网络分发、智能语音、视频监控等各类国家基建与行业应用。

云笔电

基于K1芯片，使用RISC-V笔记本电脑MUSE Book，**配置openHarmony5.0操作系统，**并成功通过移动云终端接口，流畅使用移动云上资源，依托移动云能力使国产芯片和操作系统在成本、安全和效率等方面均取得显著优势。

鸿蒙行业平板

基于K1芯片打造的全球首款RISC-V+OpenHarmony5.0原生鸿蒙解决方案，实现全栈自主创新，基于RISC-V同构融合AI算力和开源大模型，构建了新AI算力平台，可用于政企、教育、金融、医疗、交通、工控等多个行业细分场景。

机器人

空间计算模组

基于K1芯片打造的双目纯视觉空间计算模组，借助于K1高达256bit的向量计算能力，非常适配eign、superpoint等空间计算底层算法加速；产品集成了鱼眼双目广角相机、IMU模块，同时提供整合空间算法，已应用在无人机、割草机、具身机器人等产品。

智能语音模组

采用K1芯片，凭借其强悍性能为高效语音转写提供助力，场景本地实现，数据不上云端，保障数据的私密性和转写的实时性。深度算法适配，书面语转写字错误率低于0.97%，自由交谈错误率低于2.65%，可广泛应用于会议、教育以及机器人等场景。

无人机

基于K1芯片打造的ROS2无人机解决方案，实现自动避障、路径规划等核心功能，适用于物流巡检、电力巡查、测绘勘探等场景，毫秒级实时响应保障飞行安全。8 核算力支撑多传感器融合，低功耗设计延长续航，开源架构灵活适配行业需求，是高效可靠的智能空中助手。

教育机器人

搭载K1芯片，内置Ubuntu操作系统，无需搭配PC主控，连接显示器、键盘、鼠标即可使用，是快速构建机械臂编程教育、控制逻辑开发、机器人应用、ROS仿真实验课堂的首选助手，帮助用户快速开始六轴机械臂学习与应用。

未来，进迭时空将持续专注RISC-V AI CPU，以RISC-V AI CPU构建面向大模型新AI时代的最佳原生计算平台，助力RISC-V AI CPU的发展。

责编：Johnson Zhang

获奖理由： KL730芯片集成了最先进第三代KDP系列可重构NPU架构，可提供高达8TOPS的有效算力。新NPU架构不仅对最新CNN网络架构有更高运算效率，在transfomer应用方面也有很好的效果，同时极大降低了运算对 DDR 带宽需求。KL730具备强大的视频处理能力，支持4K 60FPS的输出。 凭借耐能团队在ISP领域十余年的经验沉淀，KL730在降噪、宽动态、鱼眼校正、暗光成像等方面有非常卓越的表现。广泛应用在以下领域：智能安防、智能驾驶、视频会议、工业相机、商用机器人、CMS电子后视镜。 Quad ARM® Cortex™ A55 CPU。 内建DSP，可以加速AI模型后处理，语音处理。 Linux和RTOS、TSMC 12 纳米工艺。 高达 4K@60fps分辨率，与主流传感器的无缝 RGB Bayer 接口，多达4通道影像接口。 高达3.6eTOPS@int8 / 7.2eTops@int4。 支持Cafee、Tensorflow、Tensorflowlite、Pytorch、Keras、ONNX框架。 并兼容CNN、Transformer、RNN Hybrid等多种AI模型， 有更高的处理效率和精度。

获奖理由： 墨芯人工智能S40计算卡（下文简称S40计算卡）为数据中心的AI推理应用而打造。作为通用深度学习推理加速器，外形规格采用双槽全高全长 PCIe x16金手指物理形式。S40计算卡支持80GB LPDDR4x内存，理论内存峰值带宽为256GB/s，最大功耗300W。被动冷却板设计使其在热限制内，通过系统气流来实现计算卡的操作。 S40计算卡基于墨芯人工智能Antoum®️架构构建。通过软硬件紧密结合的架构设计，强调平衡的结构化稀疏性，支持高达32倍的高稀疏率。基于Antoum®️架构，S40计算卡支持BF16和INT8计算。同时，S40计算卡支持包括集成模型稀疏器的软件工具链、编译器和运行时在内的端到端软件解决方案，确保主流AI推理作业可以快速实现。 硬件与软件紧密结合的设计使得Antoum®️成为一个高效的人工智能片上系统处理器。此外，S40计算卡还支持硬件视频编解码器和JPEG解码器，使其能够处理各种视频和图像应用场景。同时，S40计算卡随设备发货时，为系统DDR开启ECC功能，防止内存出现可检测的错误。

获奖理由： 公司核心产品是AI CPU芯片，使用RISC-V指令扩展在CPU内实现AI算力，具有AI通用性强（类似英伟达CUDA Core）、部署便捷等特点，有望成为AI CPU芯片的最佳技术路线。未来所有的端侧电子设备产品都会接入大模型，以获取智能化能力，为数十万亿规模的电子信息制造业提供增量市场和高质量发展的机遇，市场空间广阔。大模型需要在资源有限的终端设备上高效运行，对芯片的AI部署通用性、易用性、经济性等提出了更高的要求。进迭时空的实践表明，RISC-V作为开源开放的下一代计算架构，以通用CPU核为基础，结合少量DSA定制和大量微架构创新，不仅具备完整CPU功能，更拥有强大的原生AI算力，可以为端侧大模型落地应用提供全新的解决方案。 2024年，公司推出业界首款8核RISC-V AI CPU芯片SpacemiT Key Stone® K1，主频1.8GHz，CPU算力＞50 KDMIPS，CPU提供的融合AI算力2TOPS@INT8。K1的融合AI算力通用性强，可快速部署几乎所有的AI算法，包括多个5亿到30亿参数的本地大模型推理，运行5亿参数大模型可以实现每秒20个token的优异性能。荣获2024第十九届“中国芯”优秀技术创新产品奖。这是首个RISC-V高性能计算芯片荣获“中国芯”奖项。 目前，K1芯片客户37家，量产数万颗，在电网、运营商、工业、机器人、消费电子、开发者等场景完成量产应用或产品化落地。在RISC-V高算力芯片中，不仅量产速度最快，而且出货量最多, 推动了RISC-V高性能计算芯片的应用领域从开发者市场破圈，首次规模化进入了行业市场。

获奖理由： AzureBlade K340L智能加速卡以芯动力自研的AE7100芯片为核心，AE7100基于自研RPP架构设计，具有众多优势。其实现了低成本、低功耗、低延时、高性能、快速部署与广泛应用的完美平衡，结合NPU效率与GPU通用性，为AI计算带来全新方案。 芯动力AzureBlade K340L智能加速卡为边缘端AI推理应用而设计，大小仅为半张名片，标准 M.2 接口方便易用，兼容 CUDA 和 ONNX，适配各类AI应用。拥有25.6TOPs的算力和60GB/s的内存带宽，功耗仅为8W。 它支持 RPP 汇编及 CUDA C/C++，简化了应用开发流程，基于 Resnet 50 测试每秒处理1500张图像，性能远超 Nvidia Jetson Xavier，部署高效。创新扇出型封装技术，以玻璃载板降成本，提升线密度，优化散热电气，为 AI PC 提供 DNPU 方案，推动产业发展。 该加速卡还为AI PC领域带来了革新性突破。为使芯片嵌入笔记本电脑，芯动力创新封装技术，采用扇出型封装，以玻璃载板替代昂贵的 ABF 材料，实现低成本先进封装。此封装方式将线密度提升至 5 微米以下，通过三层金属线设计减小芯片面积，优化散热与电气性能。封装后的 M.2 卡为 AI PC 提供了 DNPU 解决方案，助力 AI PC 产业快速落地。 应用方面，已成功适配如Llama3-8B、Stable Diffusion、通义千问等先进模型，并在边缘大模型、工业自动化、安全监控、医疗影像及信号处理等多个领域展示了其广泛的应用潜力。

Towards Deep Learning using TensorFlow Lite on RISC-V

Marcia Sahaya Louis

marcia93@bu.edu

Boston University

Zahra Azad

zazad@bu.edu

Boston University

Leila Delshadtehrani

delshad@bu.edu

Boston University

Suyog Gupta

suyoggupta@google.com

Google Inc.

Pete Warden

petewarden@google.com

Google Inc.

Vijay Janapa Reddi

vj@eecs.harvard.edu

Harvard University

Ajay Joshi

joshi@bu.edu

Boston University

Abstract

Deep neural networks have been extensively adopted for a myr-

iad of applications due to their ability to learn patterns from large

amounts of data. The desire to preserve user privacy and reduce

user-perceived latency has created the need to perform deep neu-

ral network inference tasks on low-power consumer edge devices.

Since such tasks often tend to be computationally intensive, of-

floading this compute from mobile/embedded CPU to a purpose-

designed "Neural Processing Engines" is a commonly adopted so-

lution for accelerating deep learning computations. While these

accelerators offer significant speed-ups for key machine learning

kernels, overheads resulting from frequent host-accelerator com-

munication often diminish the net application-level benefit of this

heterogeneous system. Our solution for accelerating such work-

loads involves developing ISA extensions customized for machine

learning kernels and designing a custom in-pipeline execution unit

for these specialized instructions. We base our ISA extensions on

RISC-V: an open ISA specification that lends itself to such special-

izations. In this paper, we present the software infrastructure for

optimizing neural network execution on RISC-V with ISA exten-

sions. Our ISA extensions are derived from the RISC-V Vector ISA

proposal, and we develop optimized implementations of the critical

kernels such as convolution and matrix multiplication using these

instructions. These optimized functions are subsequently added to

the TensorFlow Lite source code and cross-compiled for RISC-V.

We find that only a small set of instruction extensions achieves

coverage over a wide variety of deep neural networks designed for

vision and speech-related tasks. On average, our software imple-

mentation using the extended instructions set reduces the executed

instruction count by 8X in comparison to baseline implementation.

In parallel, we are also working on the hardware design of the in-

pipeline machine learning accelerator. We plan to open-source our

Permission to make digital or hard copies of all or part of this work for personal or

classroom use is granted without fee provided that copies are not made or distributed

for profit or commercial advantage and that copies bear this notice and the full citation

on the first page. Copyrights for components of this work owned by others than the

author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or

republish, to post on servers or to redistribute to lists, requires prior specific permission

and/or a fee. Request permissions from permissions@acm.org.

CARRV ’19, June 22, 2019, Phoenix, AZ

© 2019 Copyright held by the owner/author(s). Publication rights licensed to ACM.

ACM ISBN 978-1-4503-9999-9/18/06...$15.00

https://doi.org/10.1145/1122445.1122456

software modifications to TF Lite, as well as the micro-architecture

design in due course.

Keywords

Deep Learning, RISC-V Vector ISA extension, TensorFlow Lite

1

Introduction

Recent developments in deep learning have led to a resurgence in

artificial intelligence. Various cognitive tasks such as image recog-

nition [19, 23], speech recognition [31], and natural language pro-

cessing [6, 20] extensively use deep neural networks. As these "in-

telligent applications" pervade into mobile/Internet of Things (IoT)

platforms, there is a growing demand for efficient execution of deep

neural networks on these low-power and resource-constrained plat-

forms. However, state-of-the-art neural networks routinely have

millions of parameters and a single inference task can invoke bil-

lions of arithmetic operations and memory accesses. Offloading

the neural network execution to a dedicated hardware accelerator

has emerged as a widely adopted solution for improving the execu-

tion time and energy efficiency. Manifestations of this concept are

abundant: the Apple A12 Bionic [27] that has an Integrated Neural

Processing Unit, the Qualcomm SD 855 that has a Hexagon DSP

[5, 12] and an integrated Neural Processing Unit, Huawei’s Kirin

980 SoC that has a Dual Neural Processing Unit [3], and Samsung

Exynos 9820, that has an integrated Neural Processing Unit [4].

A heterogeneous solution comprised of accelerators and CPU

often requires partitioning the work between the host CPU and

the neural accelerator(s) and may trigger frequent host-accelerator

communications. Consider a canonical machine learning applica-

tion that comprises of a) pre-processing the inputs to render them

consumable by a neural network, b) running a neural network

inference using these inputs, and c) post-processing the predic-

tions generated by the network. The net application-level speed-up

is determined by the relative computational complexities of the

components listed above as well as the overheads associated with

communication between the host and the accelerator. Applications

that involve frequent data and/or control exchanges between the

host and accelerator land up severely under-utilizing the accelerator

and may not see a net benefit of offloading work from the host.

In this paper, we present our work on developing a solution

that seeks to eliminate these overheads that surface in a typical

CARRV ’19, June 22, 2019, Phoenix, AZ

Louis et al.

Source Codes

Compiler

Program Execution

Intrinsics

C++ Source (TFlite)

RISC-V

GNU toolchain

Spike ISS

Processor supporting

RISC-V V ISA

Program outputs

Figure 1: Overview of the Software infrastructure. ‘Intrinsics’ are implemented using C inline assembly functions.

heterogeneous system. Our solution hinges on developing ISA ex-

tensions customized for machine learning kernels and designing

a custom in-pipeline execution unit for these specialized instruc-

tions. To explore this idea, as a first step, we developed the software

infrastructure to support custom domain specific ISA extension

for machine learning. We used the open source RISC-V ISA as our

target ISA [8, 30]. RISC-V ISA consists of a base Integer (I) ISA

which is mandatory for every RISC-V core implementation, and

optional extensions to the base ISA. The capability of ISA-level

customization provides an opportunity to specialize our processor

designs for machine learning workloads.

To effectively accelerate ML on RISC-V processors, our ISA ex-

tensions are derived from the RISC-V vector ISA proposal [22].

We selected a subset of the instructions necessary to implement

the key machine learning kernels. We developed the tool-chain

by augmenting the software environment with the right inline as-

sembly support and building the run-time that can effectively map

the high-level macros to the low-level ISA execution. We added

basic compiler support for the extended instructions using C inline

assembly functions. The C inline assembly functions are used to

implement TensorFlow Lite [1] kernel operations such as convolu-

tion and matrix multiplication. We added these optimized functions

to TensorFlow Lite source code and cross-compiled them for RISC-

V target. We modified Spike [7], an instruction set simulator, to

support the extended instructions. Subsequently, we used Spike

for functional verification and for benchmarking machine learning

models. We use the executed instruction count as the metric to

compare the modified RISC-V ISA with ARM v-8A with NEON

Advanced SIMD extensions [21].

2

Software Environment

We present our infrastructure for building TensorFlow Lite for RISC-

V target (Figure 1). As part of the software infrastructure, we have

implemented a subset of instructions from RISC-V V ISA extension

(draft v0.5) [22]. Table 1 shows the list of supported instructions.

These instructions are supported using C inline assembly functions.

We provide detailed description of modifications to the compiler

tool-chain, Spike and TensorFlow Lite in the following subsections.

2.1

Compiler support for ISA extensions

We use inline assembly functions to enable vector instruction sup-

port. The functions are known to the compiler and are mapped to

a sequence of one or more assembly instructions. For example, the

code snippet in Listing 1 shows the implementation of the vector

load template function. The function loads an array of elements to

the vector register “va1”. The number of elements to load is config-

ured at run-time by setting two Control Status Register (CSR), i.e.,

vcfg and vl as required by the RISC-V V ISA extension.

Listing 1: A function to load vector elements.

template <class T>

inline void __VectorLoadInput(const T* load_address) {

asm volatile("vls va1, 0(%0), v \t\n"

: : "r"(load_address));

}

The C inline assembly functions are compiled into assembly code

using the RISC-V GCC tool-chain. The assembly code is then con-

verted into machine code using GNU assembler (GAS) [11]. GAS is

implemented in two sections, the front-end that handles the pars-

ing of assembly code and the back-end that generates the machine

code. We added support for each of the instructions in Table 1 in

the GAS front-end to parse the extended instructions and check if

the instruction has a valid opcode and operands. Subsequently, the

GAS back-end generates the corresponding machine code for the

extended instructions. We then modified the Spike ISA simulator

to verify the functionality of the extended instructions.

2.2

Instruction simulation support on Spike ISS

Spike is a RISC-V Instruction Set Simulator (ISS) [7] and imple-

ments a functional model of RISC-V processor. Spike is a functional

simulator that ignores internal delays such as I/O accesses or mem-

ory transactions. Therefore, the simulations are not cycle accurate.

Spike executes a user space program using proxy kernel for han-

dling the system calls from a C standard library functions.

To support the simulation of the instructions in Table 1, we mod-

ified the Spike simulator. We extended the class regfile_t with

vector registers and macros to read/write values to the registers.

In order to load/store data from memory, we extended the class

mmu_t with macros for loading/storing multiple data from memory.

Similar to the scalar pipeline, a memory request is handled by the

TLB unit in Spike.

We also modified the class processor_t to configure the two

vector CSRs; vcfg CSR and vl CSR. As specified in RISC-V Vector

ISA extension [22], the vcfg CSR configures the vector unit by

Towards Deep Learning using TensorFlow Lite on RISC-V

CARRV ’19, June 22, 2019, Phoenix, AZ

Table 1: The subset of RISC-V Vector ISA extension [22] implemented in our software ecosystem.

Inst. Type

Instructions

Function

Memory access

vls{b,h,s,d} VRd, RS1, RS2, m

vlx{b,h,s,d} VRd, RS1, VRS2, m

Loads a vector into VRd from memory address in RS1

with unit/const stride in RS2 or indexed stride in VRS2

vss{b,h,s,d} VRS3, RS1, RS2, m

vsx{b,h,s,d} VRS3, RS1, VRS2, m

Stores a vector in VRS3 to memory address in RS1

with unit/const stride in RS2 or with indexed stride in VRS2

Arithmetic Instructions

vadd VRd, VRS1, VRS2, m

vmul VRd, VRS1, VRS2, m

vfadd VRd, VRS1, VRS2, m

vfmul VRd, VRS1, VRS2, m

Add/Multiply values in VRS1, VRS2 and writes to VRd

vmadd VRd, VRS1, VRS2, VRS3, m

vfmadd VRd, VRS1, VRS2, VRS3, m

Multiply values in VRS1,VRS2 and add VRS3, and writes to VRd

vmax VRd, VRS1, VRS2, m

vmin VRd, VRS1, VRS2, m

vfmax VRd, VRS1, VRS2, m

vfmin VRd, VRS1, VRS2, m

Element-wise maximum/minimum of values in VRS1, VRS2

and writes to VRd

Data Movement

vsplat VRd, VRS1, RS2, m

vbcastx VRd, RS1

vbcastf VRd, FRS1

Splats the element in VR1[RS2] to VRd

Broadcasts value in RS1/FRS1 to VRd

vredsum VRd, VRS1

vredmin VRd, VRS1

vredmax VRd, VRS1

vfredsum VRd, VRS1

Reduction of VRS1 based on sum/max/min,

broadcast and store the result to VRd

V Rd: Vector destination registers

V RS1,2,3: Vector source registers,

m: Two bit encoding for masking; m=00 -> scalar shape destination, m=01 -> unmasked vector operation, m=10 -> mask enabled where v1.LSB=0, m=11 ->

mask enabled where v1.LSB=1; here v1 is the mask register.

setting the highest number of enabled vector registers in vregmax

CSR and the maximum width of elements in vemaxw CSR. The

vl CSR holds the current active vector length. Finally, we added

support in Spike for all the instructions in Table 1. Listing 2 is

an example of implementation vadd instruction in Spike. These

modification enabled simulation of the vector instructions. We

added functionality to Spike interactive debug mode to facilitate

tracing and debugging.

Listing 2: Implementation of vadd instruction in Spike.

require_extension('V');

require_rv64;

WRITE_VRD(v_add(VRS1, VRS2, EW, insn.m(), VMASK, VL));

2.3

RISC-V target for TensorFlow Lite

TensorFlow Lite is a lightweight deep learning framework for mo-

bile and embedded devices [1]. It compresses a TensorFlow model

to a .tflite model that has a small binary size. This enables on-device

machine learning and uses hardware acceleration to improve per-

formance. The TensorFlow Lite source code has two implemen-

tations; reference_ops and optimized_ops, for machine learn-

ing kernels such as convolution and depthwise-convolution. The

reference_ops implementation is portable, hardware-independent

and uses standard C/C++ libraries. The optimized_ops is a hard-

ware specific optimized implementation of kernel operations us-

ing gemmlowp, Eigen libraries [13, 18] and other processor spe-

cific optimizations. For example, in the case of ARM processors,

the optimized_ops implementation leverages gemmlowp, Eigen

libraries and Neon instructions [21] to optimize kernel operations.

To support RISC-V target for Tensorflow Lite, we modified some

functions to remove library dependencies not supported by Newlib 1

[29] in reference_ops . This made the reference_ops implemen-

tation portable and capable of running on mobile and embedded

device with RISC-V processors. The C inline assembly functions

were used for constructing SIMD-aware optimized functions to be

used in optimized_ops implementation for RISC-V vector proces-

sors. Listing 3 shows the implementation of a function that performs

1C standard library implementation intended for use on embedded system

CARRV ’19, June 22, 2019, Phoenix, AZ

Louis et al.

Mobilenet-v1

(0.25, 128)

Mobilenet-v1

(0.5, 128)

Mobilenet-v1

(0.75, 128)

Mobilenet-v1

(1.0, 128)

0.0

0.5

1.0

1.5

2.0

2.5

# Committed Instructions

1e9

RV-base-v1

RV-base-v2

ARM-base

(a)

Mobilenet-v1

(0.25, 128)

Mobilenet-v1

(0.5, 128)

Mobilenet-v1

(0.75, 128)

Mobilenet-v1

(1.0, 128)

0

1

2

3

4

5

# Cycles

1e9

RV-base-v1

RV-base-v2

ARM-base

(b)

Figure 2: Comparison of committed instructions, cycles and IPC for ARM-base, RV-base-v1 without loop optimization and RV-

base-v2 with loop optimization for four variants of MobileNet [14]. Here, Mobilenet-v1 (0.25, 128) means MobileNet-V1 model

for input size of 128x128 pixels and 0.25 depth multiplier. The depth multiplier changes the number of channels in each layer.

element-wise addition of two arrays. Using the instructions in Table

1, we can support a wide range of machine learning models.

We cross-compiled the TensorFlow Lite source code for RISC-V

ISA and executed .tflite models on Spike. With the infrastructure in

place, we generate a binary that can run on a RISC-V processor that

has micro-architectural support for the RISC-V V ISA extension.

Listing 3: A example function for element-wise addition of

two arrays.

void VectorVectorAdd(const float* input1,

const float* input2,

float* output, int len) {

int new_len = len - (len & (kMaxVectorLength32 - 1));

int len_diff = len & (kMaxVectorLength32 - 1);

SetConfig(kElementWidthMax32, kMaxVectorLength32);

for (int i = 0; i < new_len; i += kMaxVectorLength32) {

__VectorLoad((input1 + i), (input2 + i));

__VectorAddFloat();

__VectorStore((output + i));}

if (len_diff != 0) {

SetVl(len_diff);

__VectorLoad((input1 + new_len), (input2 + new_len));

__VectorAddFloat();

__VectorStore((output + new_len));}

}

3

Evaluation

In this section, we evaluate the code optimizations for RISC-V and

compare it with ARM processors, as ARM processors are the most

commonly used processors for mobile systems. For comparison

purpose we define the Region Of Interest (ROI) as the execution of

interpreter->Invoke() function in TensorFlow Lite. The deep learning

models [14–17, 25, 26] used in our evaluation are listed in Table

2. These are commonly used machine-learning inference models

that are deployed on mobile devices. We cover a wide range of

applications using these benchmark models. The models are 32-

bit floating point .tflite models and are hosted on TensorFlow Lite

website [2].

To evaluate the performance of deep learning models listed in Ta-

ble 2 for ARM processor, we used gem5 [9] in full system mode with

ARM A-class, 4-stage pipeline High Performance In-order (HPI)

core configuration [28]. The ARM HPI was configured with 16KB

L1 I$,16KBL1D$ and without L2$ 2. In this section, we will use

the term ARM-base for the baseline implementation of TensorFlow

Lite using reference_ops, and ARM-opt for the implementation of

TensorFlow Lite using optimized_ops. We inserted m5_reset_stats

and m5_dump_stats functions in TensorFlow Lite source code to

get gem5 performance stats for ROI. We used number of cycles and

committed instructions as our performance metrics for evaluation.

For RISC-V, RV-base and RV-opt represents the RISC-V cross-

compiled binaries of TensorFlow Lite using reference_ops and

optimized_ops, respectively. We mapped a in-order 5-stage pipeline

Rocket core [7] to Zedboard [10] to evaluate the performance of

benchmarks in Table 2 for RV-base. The Rocket core is configured

with 16KB L1 I$,16KBL1D$ and without L2$, as the current ver-

sion of Rocket chip does not support L2$. We used hardware per-

formance counters, specifically the cycle CSR and instret CSR

for evaluation. Currently, the microarchitecture enhancement to

Rocket-chip processor for supporting extended instructions in Ta-

ble 1 is in the ‘pre-pre-alpha stage’. For this paper we use Spike

to benchmark number of committed instructions of deep learning

benchmarks listed in Table 2 for RV-opt.

2We simulated ARM core without L2$ to perform a fair comparison with RISC-V

Rocket core

Towards Deep Learning using TensorFlow Lite on RISC-V

CARRV ’19, June 22, 2019, Phoenix, AZ

MNasNet

DenseNet

Inception-V3

ResNet-V1

MobileNet-V1

Yolo-tiny

Speech decoder Speech encoder

10

8

10

9

10

10

# Committed Instructions

RV-base-v2

ARM-base

RV-opt-v1

ARM-opt

RV-opt-v2

Figure 3: Number of committed instructions for RV-base-v2, ARM-base, RV-opt-v1 optimized with 128bits registers, ARM-opt

and RV-opt-v2 optimized with 256bit registers for various deep learning models.

Table 2: List of deep learning models using in our evalua-

**tion. CONV = Convolution layer, LSTM = ...