uacce.rst (7177B)
1.. SPDX-License-Identifier: GPL-2.0 2 3Introduction of Uacce 4--------------------- 5 6Uacce (Unified/User-space-access-intended Accelerator Framework) targets to 7provide Shared Virtual Addressing (SVA) between accelerators and processes. 8So accelerator can access any data structure of the main cpu. 9This differs from the data sharing between cpu and io device, which share 10only data content rather than address. 11Because of the unified address, hardware and user space of process can 12share the same virtual address in the communication. 13Uacce takes the hardware accelerator as a heterogeneous processor, while 14IOMMU share the same CPU page tables and as a result the same translation 15from va to pa. 16 17:: 18 19 __________________________ __________________________ 20 | | | | 21 | User application (CPU) | | Hardware Accelerator | 22 |__________________________| |__________________________| 23 24 | | 25 | va | va 26 V V 27 __________ __________ 28 | | | | 29 | MMU | | IOMMU | 30 |__________| |__________| 31 | | 32 | | 33 V pa V pa 34 _______________________________________ 35 | | 36 | Memory | 37 |_______________________________________| 38 39 40 41Architecture 42------------ 43 44Uacce is the kernel module, taking charge of iommu and address sharing. 45The user drivers and libraries are called WarpDrive. 46 47The uacce device, built around the IOMMU SVA API, can access multiple 48address spaces, including the one without PASID. 49 50A virtual concept, queue, is used for the communication. It provides a 51FIFO-like interface. And it maintains a unified address space between the 52application and all involved hardware. 53 54:: 55 56 ___________________ ________________ 57 | | user API | | 58 | WarpDrive library | ------------> | user driver | 59 |___________________| |________________| 60 | | 61 | | 62 | queue fd | 63 | | 64 | | 65 v | 66 ___________________ _________ | 67 | | | | | mmap memory 68 | Other framework | | uacce | | r/w interface 69 | crypto/nic/others | |_________| | 70 |___________________| | 71 | | | 72 | register | register | 73 | | | 74 | | | 75 | _________________ __________ | 76 | | | | | | 77 ------------- | Device Driver | | IOMMU | | 78 |_________________| |__________| | 79 | | 80 | V 81 | ___________________ 82 | | | 83 -------------------------- | Device(Hardware) | 84 |___________________| 85 86 87How does it work 88---------------- 89 90Uacce uses mmap and IOMMU to play the trick. 91 92Uacce creates a chrdev for every device registered to it. New queue is 93created when user application open the chrdev. The file descriptor is used 94as the user handle of the queue. 95The accelerator device present itself as an Uacce object, which exports as 96a chrdev to the user space. The user application communicates with the 97hardware by ioctl (as control path) or share memory (as data path). 98 99The control path to the hardware is via file operation, while data path is 100via mmap space of the queue fd. 101 102The queue file address space: 103 104:: 105 106 /** 107 * enum uacce_qfrt: qfrt type 108 * @UACCE_QFRT_MMIO: device mmio region 109 * @UACCE_QFRT_DUS: device user share region 110 */ 111 enum uacce_qfrt { 112 UACCE_QFRT_MMIO = 0, 113 UACCE_QFRT_DUS = 1, 114 }; 115 116All regions are optional and differ from device type to type. 117Each region can be mmapped only once, otherwise -EEXIST returns. 118 119The device mmio region is mapped to the hardware mmio space. It is generally 120used for doorbell or other notification to the hardware. It is not fast enough 121as data channel. 122 123The device user share region is used for share data buffer between user process 124and device. 125 126 127The Uacce register API 128---------------------- 129 130The register API is defined in uacce.h. 131 132:: 133 134 struct uacce_interface { 135 char name[UACCE_MAX_NAME_SIZE]; 136 unsigned int flags; 137 const struct uacce_ops *ops; 138 }; 139 140According to the IOMMU capability, uacce_interface flags can be: 141 142:: 143 144 /** 145 * UACCE Device flags: 146 * UACCE_DEV_SVA: Shared Virtual Addresses 147 * Support PASID 148 * Support device page faults (PCI PRI or SMMU Stall) 149 */ 150 #define UACCE_DEV_SVA BIT(0) 151 152 struct uacce_device *uacce_alloc(struct device *parent, 153 struct uacce_interface *interface); 154 int uacce_register(struct uacce_device *uacce); 155 void uacce_remove(struct uacce_device *uacce); 156 157uacce_register results can be: 158 159a. If uacce module is not compiled, ERR_PTR(-ENODEV) 160 161b. Succeed with the desired flags 162 163c. Succeed with the negotiated flags, for example 164 165 uacce_interface.flags = UACCE_DEV_SVA but uacce->flags = ~UACCE_DEV_SVA 166 167 So user driver need check return value as well as the negotiated uacce->flags. 168 169 170The user driver 171--------------- 172 173The queue file mmap space will need a user driver to wrap the communication 174protocol. Uacce provides some attributes in sysfs for the user driver to 175match the right accelerator accordingly. 176More details in Documentation/ABI/testing/sysfs-driver-uacce.