Post

AXIDMA - ZCU102 CCI 400

Posted Updated
Preview Image
By Sanjaybyranna
Hits
23 min read
AXIDMA - ZCU102 CCI 400

This post is follow up to the previous blog - AXIDMA Loopback Test - ZCU102. Here we will explore steps to build axidma hardware with cci-400 for hardware cache coherency and will also explore kernel driver and userspace application to test.

1
2
3
4
5
6
7
8
9

# Open Vivado and use tcl console
# Navigate to home directory and verify path

cd "[file normalize ~]"
puts "Current working directory: [pwd]"

# Create and navigate to the working directory
file mkdir "[file normalize ~]/axidma-vivado"
cd "[file normalize ~]/axidma-vivado"

Getting the TCL script and building a hardware project will be same as previous project. Download the zcu102-axi-hpc-cci-400.tcl, and save it inside ~/axidma-vivado.

1
2
3
4
5
6

# make sure to save the downloaded tcl script inside this folder ~/axidma-vivado
# source the .tcl file in vivado tcl console
# this also automatically opens the project folder ~/axidma-vivado/axi-dma
source "[file normalize ~]/axidma-vivado/zcu102-axi-hpc-cci-400.tcl"
# Open the block design
open_bd_design "[file normalize ~]/axidma-vivado/zcu102-axi-hpc-cci-400/zcu102-axi-hpc-cci-400.srcs/sources_1/bd/design_1/design_1.bd"

The block diagram would look similar to this;

cci-400

1
2
3
4
5
6
7
8
9
10
11
12

# Run synthesis
launch_runs synth_1 -jobs 2
wait_on_run synth_1
# Run implementation
launch_runs impl_1 -jobs 2
wait_on_run impl_1
# Generate bitstream
launch_runs impl_1 -to_step write_bitstream -jobs 2
wait_on_run impl_1

# Export the Hardware Platform (XSA)
write_hw_platform -fixed -include_bit -force -file "[file normalize ~]/axidma-vivado/zcu102-axi-hpc-cci-400/design_1_wrapper.xsa"

ZCU102 Petalinux Build

We need few pre-requisites for petalinux build to test the axidma, which includes downloading the offline files of petalinux downloads, sstate-cache for aarch64 and petalinux toolchain.

Follow the steps explained in petalinux-starter-guide.

1
2
3
4
5
6
7
8
9
10
11
12

# open ubuntu bash terminal
cd $HOME
# Source petalinux tool chain
source $HOME/petalinux_downloads_sstate_shared/tools/petalinux/2022.2/settings.sh

# create zcu102 petalinux project
petalinux-create -t project --template zynqMP -n zcu102_axi_dma
cd zcu102_axi_dma

petalinux-config --get-hw-description=$HOME/axidma-vivado/zcu102-axi-hpc-cci-400/design_1_wrapper.xsa
# this opens a blue menuconfig window and change the following config variables
Petalinux Config Variables (15 lines)
Configuration OptionValueDescription / Purpose
CONFIG_SUBSYSTEM_BOOTARGS_AUTOyAutomatically generates kernel boot arguments
CONFIG_SUBSYSTEM_BOOTARGS_EARLYPRINTKyEnables early printk in boot arguments
CONFIG_SUBSYSTEM_EXTRA_BOOTARGScma=1024MAppends custom extra arguments (e.g., CMA memory allocation)
CONFIG_SUBSYSTEM_ROOTFS_EXT4yConfigures the root filesystem type as EXT4
SUBSYSTEM_SDROOT_DEV/dev/sda1Specifies the SD/storage root device partition
CONFIG_SUBSYSTEM_HOSTNAMEzcu102_axi_dmaSets the target board hostname
CONFIG_TMP_DIR_LOCATION${PROOT}/build/tmpDefines the Yocto temporary build directory
CONFIG_PRE_MIRROR_URLfile:///home/user/petalinux_downloads_sstate_shared/downloadsPoints to local source downloads mirror
CONFIG_YOCTO_LOCAL_SSTATE_FEEDS_URL/home/user/petalinux_downloads_sstate_shared/aarch64Specifies the local shared sstate feed path
CONFIG_YOCTO_NETWORK_SSTATE_FEEDSnDisables network-based sstate feeds
CONFIG_YOCTO_BB_NO_NETWORKnForces offline BitBake builds with internet access

After setting above configs, CONFIG_SUBSYSTEM_USER_CMDLINE will contain final kernel bootargs, which will look like earlycon console=ttyPS0,115200 clk_ignore_unused root=/dev/sda1 rw rootwait cma=1024M

Kernel Configs

1
2

# set the kernel configuration options using 
petalinux-config -c kernel
Petalinux Kernel Config Variales (22 lines)
Kernel Configuration OptionValueDescription / Purpose
CONFIG_DMATESTmEnables the generic DMA test client as a loadable module
CONFIG_XILINX_ZYNQMP_DMAyEnables the Xilinx ZynqMP DMA driver
CONFIG_XILINX_DMATESTmEnables the Xilinx DMA test client as a loadable module
CONFIG_XILINX_VDMATESTmEnables the Xilinx Video DMA test client as a loadable module
CONFIG_CMA_SIZE_MBYTES1024Sets the Contiguous Memory Allocator (CMA) size to 1024 MB
CONFIG_XILINX_DMAyEnables the core Xilinx DMA engine driver
CONFIG_ARM64_4K_PAGES4KBConfigures the ARM64 kernel page size to 4KB
CONFIG_CMAyEnables the Contiguous Memory Allocator subsystem
CONFIG_PARTITION_ADVANCEDyEnables advanced partition table types support
CONFIG_EFI_PARTITIONyEnables GPT (EFI) partition table support
CONFIG_USB_DWC3yEnables DesignWare USB3 DRD controller driver
CONFIG_USB_DWC3_DUAL_ROLEySets USB DWC3 controller to dual-role (Host/Device) mode
CONFIG_USB_SUPPORTyEnables core USB support in the kernel
CONFIG_USB_COMMONyEnables common USB framework files
CONFIG_USB_ARCH_HAS_HCDyIndicates architecture has USB Host Controller Driver support
CONFIG_USByEnables support for USB
CONFIG_USB_XHCI_HCDyEnables xHCI USB controller support (USB 3.0)
CONFIG_USB_XHCI_PLATFORMyEnables generic platform xHCI controller driver

Rootfs Configs

1
2

# set the rootfs configuration options 
petalinux-config -c rootfs
Petalinux Rootfs Config Variables (8 lines)
RootFS Configuration Option / RecipeValueDescription / Purpose
CONFIG_libmali-xlnxyEnables the Xilinx Mali GPU user-space library
CONFIG_mali-backend-waylandyConfigures the Mali graphics backend for Wayland
CONFIG_packagegroup-petalinux-westonyIncludes the PetaLinux Weston (Wayland compositor) package group
CONFIG_auto-loginyEnables automatic console login for the root user

Device tree Overlay

1
2
3

# Build the device tree yocto task to updat the DTSI files
# This will get us system-user.dtsi and pl.dtsi
petalinux-build -c device-tree

Using this command the DMA IP block parameters are succesfully translated to devicetree node, it captures the memory mapped address, scatter gather settings, buffer length, read and write channel properties.

pl.dtsi (46 lines) 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42

/*
 * CAUTION: This file is automatically generated by Xilinx.
 * Version: XSCT 
 * Today is: Mon Sep 28 11:09:27 2026
 */
 
 
/ {
  amba_pl: amba_pl@0 {
    #address-cells = <2>;
    #size-cells = <2>;
    compatible = "simple-bus";
    ranges ;
    axi_dma_0: dma@a0000000 {
      #dma-cells = <1>;
      clock-names = "s_axi_lite_aclk", "m_axi_sg_aclk", "m_axi_mm2s_aclk", "m_axi_s2mm_aclk";
      clocks = <&zynqmp_clk 71>, <&zynqmp_clk 71>, <&zynqmp_clk 71>, <&zynqmp_clk 71>;
      compatible = "xlnx,axi-dma-7.1", "xlnx,axi-dma-1.00.a";
      interrupt-names = "mm2s_introut", "s2mm_introut";
      interrupt-parent = <&gic>;
      interrupts = <0 89 4 0 90 4>;
      reg = <0x0 0xa0000000 0x0 0x10000>;
      xlnx,addrwidth = <0x40>;
      xlnx,include-sg ;
      xlnx,sg-length-width = <0x1a>;
      dma-channel@a0000000 {
        compatible = "xlnx,axi-dma-mm2s-channel";
        dma-channels = <0x1>;
        interrupts = <0 89 4>;
        xlnx,datawidth = <0x80>;
        xlnx,device-id = <0x0>;
      };
      dma-channel@a0000030 {
        compatible = "xlnx,axi-dma-s2mm-channel";
        dma-channels = <0x1>;
        interrupts = <0 90 4>;
        xlnx,datawidth = <0x80>;
        xlnx,device-id = <0x0>;
      };
    };
  };
};

1
2
3
4

# edit the system-user.dtsi to enable usb host mode and axidmatest node and also 
# our custom driver to test the dma from userspace

nano project-spec/meta-user/recipes-bsp/device-tree/files/system-user.dtsi
system-user.dtsi (71 lines) 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67

/include/ "system-conf.dtsi"
#include <dt-bindings/clock/xlnx-zynqmp-clk.h>

/ {
    /* this node is for axidmatest driver from kernel source tree */
    dmatest_0: dmatest@0 {
        compatible = "xlnx,axi-dma-test-1.00.a";
        dmas = <&axi_dma_0 0 &axi_dma_0 1>;
        dma-names = "axidma0", "axidma1";
    };

    /* this node is for custom kernel driver
     * https://github.com/Sanjay0302/axidma-user.git
     */
    axidma_user_0: axidma_user@0 {
        compatible = "xlnx,axidma-user-1.0";
        dmas = <&axi_dma_0 0 &axi_dma_0 1>;
        dma-names = "axidma0", "axidma1";
    };
    
    /* dma-coherent is used only when we have HPC (cci-400) slave interface
     * currently dma-coherent property has an error, so this property is
     * currently commented;
     * the issue is being discussed in this query: 
     * https://adaptivesupport.amd.com/s/question/0D5Pd00001wgFC1KAM/im-trying-to-use-hpc0fpd-port-for-my-axi-dma-interface-and-im-facing-issue-with-dmacoherrent?language=en_US
     */

    /* this node is for custom kernel driver
     * https://github.com/Sanjay0302/software-prototypes/tree/master/linux-user-space-dma/Software
     */
    dma_proxy {  
        compatible = "xlnx,dma_proxy";
        dmas = <&axi_dma_0 0 &axi_dma_0 1>;
        dma-names = "dma_proxy_tx", "dma_proxy_rx";  
        // dma-coherent;
    };

    /* every time after changing the pl clk this has to be manually updated to
     * that clk frequency, else it will by default use 99.9MHz
     */
    fclk0 {
        status = "okay";
        assigned-clocks = <&zynqmp_clk PL0_REF>;
        assigned-clock-rates = <200000000>; // 200MHz
    };
};

/* some driver expect the channel id to be unique */
&axi_dma_0 {
    dma-channel@a0000030 {
        xlnx,device-id = <1>;
    };
    // dma-coherent;
    /* uncomment dma-coherent after solving the issue discussed in://
     * https://adaptivesupport.amd.com/s/question/0D5Pd00001wgFC1KAM/im-trying-to-use-hpc0fpd-port-for-my-axi-dma-interface-and-im-facing-issue-with-dmacoherrent?language=en_US
     */
};

/* enable usb host mode to use the usb as boot media */
&usb0 {
    status = "okay";
};

&dwc3_0 {
    status = "okay";
    dr_mode = "host";
};

Adding Driver Recipes

We have 2 kernel-module recipes, dma-proxy driver is a existing tested driver from blog - Linux DMA From User Space 2.0, and axidma-user is another custom driver with custom implementation.

First will add dma-proxy to recipes-modules.

1
2
3
4
5
6
7
8
9
10
11

# this will create the following structure inside the meta-user/recipes-modules
petalinux-create -t modules --name dma-proxy --enable
# . meta-user/recipes-modules/dma-proxy
# ├── dma-proxy.bb
# ├── files
# │   └── dma-proxy.c
# └── README
#
# remove auto created files/dma-proxy.c and replace the contents of dma-proxy.bb
rm meta-user/recipes-module/dma-proxy/files/dma-proxy.c
# and make sure CONFIG_dma-proxy is enabled in `petalinux-config -c rootfs`

Now replace the content of dma-proxy.bb with, dma-proxy.bb

Next will add axidma-user to recipes-modules.

1
2
3
4
5
6
7
8
9
10
11

# this will create the following structure inside the meta-user/recipes-modules
petalinux-create -t modules --name axidma-user --enable
# . meta-user/recipes-modules/axidma-user
# ├── axidma-user.bb
# ├── files
# │   └── axidma-user.c
# └── README
#
# remove auto created files/axidma-user.c and replace the contents of axidma-user.bb
rm meta-user/recipes-module/axidma-user/files/axidma-user.c
# and make sure CONFIG_axidma-user is enabled in `petalinux-config -c rootfs`

Now replace the content of axidma-user.bb with, axidma-user.bb

Build and Flash

Since these kernel-modules recipes uses git remote url as SRC_URI, make sure that CONFIG_YOCTO_BB_NO_NETWORK is disabled.

1
2
3

petalinux-build -c device-tree
petalinux-build -c dma-proxy
petalinux-build -c axidma-user

Follow the same exact steps we discussed in

Once you are in u-boot hush shell, will boot kernel from usb flash drive manually

1

ext4load usb 0:1 0x00200000 /boot/Image; ext4load usb 0:1 0x00100000 /boot/system.dtb; setenv bootargs "console=ttyPS0,115200 root=/dev/sda1 rw rootwait cma=1024M"; booti 0x00200000 - 0x00100000

Testing dma-proxy driver

Below Test is done with dma-coherent property disabled in the device tree.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20

## With dma-coherent property removed

# for this command u can also use 
root@zcu102axidma:~# modprobe dma_proxy  internal_test=1
[  534.696020] Starting internal test
[  534.699597] Internal test complete

root@zcu102axidma:~# time dma-proxy-test 1024000 1
DMA proxy test
Verify = 0
TX#0: ffff7f8f9000
RX#0: ffff7f4f8000
Time: 17795119 microseconds
Transfer size: 1024000 KB
Throughput: 58 MB / sec 
DMA proxy test complete
 
real    0m17.798s
user    0m1.361s
sys     0m14.670s

Testing axidma-user driver

Even this driver is tested with dma-coherent disabled.

Try running the same driver once the known issue has been solved, follow this forum for the updates.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27

root@zcu102axidma:~# dd if=/dev/urandom of=test_1mb_in.bin bs=1M count=1
1+0 records in
1+0 records out

root@zcu102axidma:~# dd if=/dev/urandom of=test_100mb_in.bin bs=1M count=100
100+0 records in
100+0 records out

root@zcu102axidma:~# dd if=/dev/urandom of=test_72mb_in.bin bs=1M count=72 
72+0 records in
72+0 records out

root@zcu102axidma:~# dd if=/dev/urandom of=test_1000mb_in.bin bs=1M count=1000
1000+0 records in
1000+0 records out

root@zcu102axidma:~# ls -la
total 1201172
drwx------    2 root     root          4096 Nov 19 17:21 .
drwxr-xr-x    5 root     root          4096 Mar  9  2018 ..
-rw-r--r--    1 root     root     1048576000 Nov 19 17:22 test_1000mb_in.bin
-rw-r--r--    1 root     root     104857600 Nov 19 17:20 test_100mb_in.bin
-rw-r--r--    1 root     root       1048576 Nov 19 17:20 test_1mb_in.bin
-rw-r--r--    1 root     root      75497472 Nov 19 17:20 test_72mb_in.bin

# Mount the tmpfs to store the received bin file to avoid flash memory bottleneck while testing the driver.
root@zcu102axidma:~# mount -t tmpfs tmpfs /tmp
1MB test result @200MHz (21 lines) 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17

root@zcu102axidma:~# axidma_example test_1mb_in.bin /tmp/test_1mb_in.bin 
[  221.663510] axidma_user axidma_user@0: TX eventfd registered (efd=5)
[  221.669953] axidma_user axidma_user@0: RX eventfd registered (efd=6)
[axidma] opened: TX 4 slots x 32 MiB, RX 4 slots x 32 MiB
[TX] sent chunk 1: slot=0 len=1048576 total=1048576 bytes
[  221.706617] axidma_user: [TX DMA] slot 0: 1048576 bytes in 334 us (2994 MB/s, 25.11 Gbps)
[  221.719642] axidma_user: [RX DMA] slot 0: 1048576 bytes in 43306 us (23 MB/s, 0.19 Gbps)
[RX] received chunk 1: slot=0 len=1048576 total=1048576 bytes
[main] transfer complete: 1048576 bytes in 0.0657 s (15.21 MB/s, 0.13 Gbps)

# Compare the files for diffrence
root@zcu102axidma:~# diff test_1mb_in.bin /tmp/test_1mb_in.bin 

# Dont let the test files fill up the tmpfs storage (RAM)
root@zcu102axidma:~# rm /tmp/test_1mb_in.bin 

root@zcu102axidma:~# sync
72MB test result @200MHz (27 lines) 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23

root@zcu102axidma:~# axidma_example test_72mb_in.bin /tmp/test_72mb_in.bin 
[  289.662355] axidma_user axidma_user@0: TX eventfd registered (efd=5)
[  289.668755] axidma_user axidma_user@0: RX eventfd registered (efd=6)
[axidma] opened: TX 4 slots x 32 MiB, RX 4 slots x 32 MiB
[TX] sent chunk 1: slot=1 len=33554432 total=33554432 bytes
[  289.718809] axidma_user: [TX DMA] slot 1: 33554432 bytes in 10428 us (3068 MB/s, 25.74 Gbps)
[  289.727301] axidma_user: [RX DMA] slot 1: 33554432 bytes in 67999547 us (0 MB/s, 0.00 Gbps)
[TX] sent chunk 2: slot=2 len=33554432 total=67108864 bytes
[  289.755637] axidma_user: [TX DMA] slot 2: 33554432 bytes in 10636 us (3008 MB/s, 25.23 Gbps)
[  289.764145] axidma_user: [RX DMA] slot 2: 33554432 bytes in 28457 us (1124 MB/s, 9.43 Gbps)
[TX] sent chunk 3: slot=3 len=8388608 total=75497472 bytes
[  289.775018] axidma_user: [TX DMA] slot 3: 8388608 bytes in 10909 us (733 MB/s, 6.15 Gbps)
[  289.785892] axidma_user: [RX DMA] slot 3: 8388608 bytes in 13385 us (597 MB/s, 5.01 Gbps)
[RX] received chunk 1: slot=1 len=33554432 total=33554432 bytes
[RX] received chunk 2: slot=2 len=33554432 total=67108864 bytes
[RX] received chunk 3: slot=3 len=8388608 total=75497472 bytes
[main] transfer complete: 75497472 bytes in 1.1810 s (60.97 MB/s, 0.51 Gbps)

root@zcu102axidma:~# diff test_72mb_in.bin /tmp/test_72mb_in.bin 

root@zcu102axidma:~# rm /tmp/test_72mb_in.bin 

root@zcu102axidma:~# sync
100MB test result @200MHz (31 lines) 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27

root@zcu102axidma:~# axidma_example test_100mb_in.bin /tmp/test_100mb_in.bin 
[  327.322348] axidma_user axidma_user@0: TX eventfd registered (efd=5)
[  327.328757] axidma_user axidma_user@0: RX eventfd registered (efd=6)
[axidma] opened: TX 4 slots x 32 MiB, RX 4 slots x 32 MiB
[TX] sent chunk 1: slot=0 len=33554432 total=33554432 bytes
[  327.378189] axidma_user: [TX DMA] slot 0: 33554432 bytes in 10452 us (3061 MB/s, 25.68 Gbps)
[  327.386682] axidma_user: [RX DMA] slot 0: 33554432 bytes in 37592592 us (0 MB/s, 0.00 Gbps)
[TX] sent chunk 2: slot=1 len=33554432 total=67108864 bytes
[  327.414677] axidma_user: [TX DMA] slot 1: 33554432 bytes in 10655 us (3003 MB/s, 25.19 Gbps)
[  327.423168] axidma_user: [RX DMA] slot 1: 33554432 bytes in 28081 us (1139 MB/s, 9.55 Gbps)
[TX] sent chunk 3: slot=2 len=33554432 total=100663296 bytes
[TX] sent chunk 4: slot=3 len=4194304 total=104857600 bytes
[  327.453669] axidma_user: [TX DMA] slot 2: 33554432 bytes in 10567 us (3028 MB/s, 25.40 Gbps)
[  327.464225] axidma_user: [RX DMA] slot 2: 33554432 bytes in 32668 us (979 MB/s, 8.21 Gbps)
[  327.473726] axidma_user: [TX DMA] slot 3: 4194304 bytes in 9528 us (419 MB/s, 3.52 Gbps)
[  327.481844] axidma_user: [RX DMA] slot 3: 4194304 bytes in 9346 us (427 MB/s, 3.59 Gbps)
[RX] received chunk 1: slot=0 len=33554432 total=33554432 bytes
[RX] received chunk 2: slot=1 len=33554432 total=67108864 bytes
[RX] received chunk 3: slot=2 len=33554432 total=100663296 bytes
[RX] received chunk 4: slot=3 len=4194304 total=104857600 bytes
[main] transfer complete: 104857600 bytes in 1.6222 s (61.64 MB/s, 0.52 Gbps)

root@zcu102axidma:~# diff test_100mb_in.bin /tmp/test_100mb_in.bin 

root@zcu102axidma:~# rm /tmp/test_100mb_in.bin 

root@zcu102axidma:~# sync
1GB test result @200MHz (143 lines) 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139

root@zcu102axidma:~# axidma_example test_1000mb_in.bin /tmp/test_1000mb_in.bin 
[  755.278407] axidma_user axidma_user@0: TX eventfd registered (efd=5)
[  755.284817] axidma_user axidma_user@0: RX eventfd registered (efd=6)
[axidma] opened: TX 4 slots x 32 MiB, RX 4 slots x 32 MiB
[TX] sent chunk 1: slot=0 len=33554432 total=33554432 bytes
[  755.334847] axidma_user: [TX DMA] slot 0: 33554432 bytes in 10415 us (3072 MB/s, 25.77 Gbps)
[  755.343358] axidma_user: [RX DMA] slot 0: 33554432 bytes in 427437820 us (0 MB/s, 0.00 Gbps)
[TX] sent chunk 2: slot=1 len=33554432 total=67108864 bytes
[  755.371527] axidma_user: [TX DMA] slot 1: 33554432 bytes in 10667 us (2999 MB/s, 25.16 Gbps)
[  755.380044] axidma_user: [RX DMA] slot 1: 33554432 bytes in 28206 us (1134 MB/s, 9.51 Gbps)
[TX] sent chunk 3: slot=2 len=33554432 total=100663296 bytes
[  755.410904] axidma_user: [TX DMA] slot 2: 33554432 bytes in 10656 us (3003 MB/s, 25.19 Gbps)
[  755.419402] axidma_user: [RX DMA] slot 2: 33554432 bytes in 30963 us (1033 MB/s, 8.66 Gbps)
[TX] sent chunk 4: slot=3 len=33554432 total=134217728 bytes
[  755.450572] axidma_user: [TX DMA] slot 3: 33554432 bytes in 10829 us (2955 MB/s, 24.78 Gbps)
[  755.459068] axidma_user: [RX DMA] slot 3: 33554432 bytes in 31283 us (1022 MB/s, 8.58 Gbps)
[TX] sent chunk 5: slot=0 len=33554432 total=167772160 bytes
[TX] sent chunk 6: slot=1 len=33554432 total=201326592 bytes
[TX] sent chunk 7: slot=2 len=33554432 total=234881024 bytes
[TX] sent chunk 8: slot=3 len=33554432 total=268435456 bytes
[RX] received chunk 1: slot=0 len=33554432 total=33554432 bytes
[  755.894026] axidma_user: [TX DMA] slot 0: 33554432 bytes in 429472 us (74 MB/s, 0.62 Gbps)
[  755.902331] axidma_user: [RX DMA] slot 0: 33554432 bytes in 18483 us (1731 MB/s, 14.52 Gbps)
[TX] sent chunk 9: slot=0 len=33554432 total=301989888 bytes
[RX] received chunk 2: slot=1 len=33554432 total=67108864 bytes
[  756.396021] axidma_user: [TX DMA] slot 1: 33554432 bytes in 493713 us (64 MB/s, 0.54 Gbps)
[  756.404337] axidma_user: [RX DMA] slot 1: 33554432 bytes in 18561 us (1724 MB/s, 14.46 Gbps)
[TX] sent chunk 10: slot=1 len=33554432 total=335544320 bytes
[RX] received chunk 3: slot=2 len=33554432 total=100663296 bytes
[  756.897973] axidma_user: [TX DMA] slot 2: 33554432 bytes in 493669 us (64 MB/s, 0.54 Gbps)
[  756.906269] axidma_user: [RX DMA] slot 2: 33554432 bytes in 18487 us (1730 MB/s, 14.52 Gbps)
[TX] sent chunk 11: slot=2 len=33554432 total=369098752 bytes
[RX] received chunk 4: slot=3 len=33554432 total=134217728 bytes
[  757.399930] axidma_user: [TX DMA] slot 3: 33554432 bytes in 493680 us (64 MB/s, 0.54 Gbps)
[  757.408235] axidma_user: [RX DMA] slot 3: 33554432 bytes in 18547 us (1725 MB/s, 14.47 Gbps)
[TX] sent chunk 12: slot=3 len=33554432 total=402653184 bytes
[RX] received chunk 5: slot=0 len=33554432 total=167772160 bytes
[  757.901950] axidma_user: [TX DMA] slot 0: 33554432 bytes in 493736 us (64 MB/s, 0.54 Gbps)
[  757.910248] axidma_user: [RX DMA] slot 0: 33554432 bytes in 18486 us (1731 MB/s, 14.52 Gbps)
[TX] sent chunk 13: slot=0 len=33554432 total=436207616 bytes
[RX] received chunk 6: slot=1 len=33554432 total=201326592 bytes
[  758.403836] axidma_user: [TX DMA] slot 1: 33554432 bytes in 493610 us (64 MB/s, 0.54 Gbps)
[  758.412142] axidma_user: [RX DMA] slot 1: 33554432 bytes in 18553 us (1724 MB/s, 14.46 Gbps)
[TX] sent chunk 14: slot=1 len=33554432 total=469762048 bytes
[RX] received chunk 7: slot=2 len=33554432 total=234881024 bytes
[  758.905640] axidma_user: [TX DMA] slot 2: 33554432 bytes in 493520 us (64 MB/s, 0.54 Gbps)
[  758.913947] axidma_user: [RX DMA] slot 2: 33554432 bytes in 18509 us (1728 MB/s, 14.50 Gbps)
[TX] sent chunk 15: slot=2 len=33554432 total=503316480 bytes
[RX] received chunk 8: slot=3 len=33554432 total=268435456 bytes
[  759.407609] axidma_user: [TX DMA] slot 3: 33554432 bytes in 493691 us (64 MB/s, 0.54 Gbps)
[  759.415911] axidma_user: [RX DMA] slot 3: 33554432 bytes in 18539 us (1726 MB/s, 14.47 Gbps)
[TX] sent chunk 16: slot=3 len=33554432 total=536870912 bytes
[RX] received chunk 9: slot=0 len=33554432 total=301989888 bytes
[  759.909636] axidma_user: [TX DMA] slot 0: 33554432 bytes in 493746 us (64 MB/s, 0.54 Gbps)
[  759.917933] axidma_user: [RX DMA] slot 0: 33554432 bytes in 18511 us (1728 MB/s, 14.50 Gbps)
[TX] sent chunk 17: slot=0 len=33554432 total=570425344 bytes
[RX] received chunk 10: slot=1 len=33554432 total=335544320 bytes
[  760.411565] axidma_user: [TX DMA] slot 1: 33554432 bytes in 493653 us (64 MB/s, 0.54 Gbps)
[  760.419861] axidma_user: [RX DMA] slot 1: 33554432 bytes in 18543 us (1725 MB/s, 14.47 Gbps)
[TX] sent chunk 18: slot=1 len=33554432 total=603979776 bytes
[RX] received chunk 11: slot=2 len=33554432 total=369098752 bytes
[  760.914713] axidma_user: [TX DMA] slot 2: 33554432 bytes in 494873 us (64 MB/s, 0.54 Gbps)
[  760.923016] axidma_user: [RX DMA] slot 2: 33554432 bytes in 18535 us (1726 MB/s, 14.48 Gbps)
[TX] sent chunk 19: slot=2 len=33554432 total=637534208 bytes
[RX] received chunk 12: slot=3 len=33554432 total=402653184 bytes
[  761.418164] axidma_user: [TX DMA] slot 3: 33554432 bytes in 495174 us (64 MB/s, 0.54 Gbps)
[  761.426470] axidma_user: [RX DMA] slot 3: 33554432 bytes in 18509 us (1728 MB/s, 14.50 Gbps)
[TX] sent chunk 20: slot=3 len=33554432 total=671088640 bytes
[RX] received chunk 13: slot=0 len=33554432 total=436207616 bytes
[  761.919575] axidma_user: [TX DMA] slot 0: 33554432 bytes in 493127 us (64 MB/s, 0.54 Gbps)
[  761.927875] axidma_user: [RX DMA] slot 0: 33554432 bytes in 18527 us (1727 MB/s, 14.48 Gbps)
[TX] sent chunk 21: slot=0 len=33554432 total=704643072 bytes
[RX] received chunk 14: slot=1 len=33554432 total=469762048 bytes
[  762.420987] axidma_user: [TX DMA] slot 1: 33554432 bytes in 493133 us (64 MB/s, 0.54 Gbps)
[  762.429290] axidma_user: [RX DMA] slot 1: 33554432 bytes in 18533 us (1726 MB/s, 14.48 Gbps)
[TX] sent chunk 22: slot=1 len=33554432 total=738197504 bytes
[RX] received chunk 15: slot=2 len=33554432 total=503316480 bytes
[  762.922336] axidma_user: [TX DMA] slot 2: 33554432 bytes in 493066 us (64 MB/s, 0.54 Gbps)
[  762.930637] axidma_user: [RX DMA] slot 2: 33554432 bytes in 18501 us (1729 MB/s, 14.50 Gbps)
[TX] sent chunk 23: slot=2 len=33554432 total=771751936 bytes
[RX] received chunk 16: slot=3 len=33554432 total=536870912 bytes
[  763.424256] axidma_user: [TX DMA] slot 3: 33554432 bytes in 493640 us (64 MB/s, 0.54 Gbps)
[  763.432562] axidma_user: [RX DMA] slot 3: 33554432 bytes in 18558 us (1724 MB/s, 14.46 Gbps)
[TX] sent chunk 24: slot=3 len=33554432 total=805306368 bytes
[RX] received chunk 17: slot=0 len=33554432 total=570425344 bytes
[  763.925472] axidma_user: [TX DMA] slot 0: 33554432 bytes in 492937 us (64 MB/s, 0.54 Gbps)
[  763.933773] axidma_user: [RX DMA] slot 0: 33554432 bytes in 18503 us (1729 MB/s, 14.50 Gbps)
[TX] sent chunk 25: slot=0 len=33554432 total=838860800 bytes
[RX] received chunk 18: slot=1 len=33554432 total=603979776 bytes
[  764.426847] axidma_user: [TX DMA] slot 1: 33554432 bytes in 493095 us (64 MB/s, 0.54 Gbps)
[  764.435154] axidma_user: [RX DMA] slot 1: 33554432 bytes in 18546 us (1725 MB/s, 14.47 Gbps)
[TX] sent chunk 26: slot=1 len=33554432 total=872415232 bytes
[RX] received chunk 19: slot=2 len=33554432 total=637534208 bytes
[  764.930770] axidma_user: [TX DMA] slot 2: 33554432 bytes in 495646 us (64 MB/s, 0.54 Gbps)
[  764.939073] axidma_user: [RX DMA] slot 2: 33554432 bytes in 18536 us (1726 MB/s, 14.48 Gbps)
[TX] sent chunk 27: slot=2 len=33554432 total=905969664 bytes
[RX] received chunk 20: slot=3 len=33554432 total=671088640 bytes
[  765.434640] axidma_user: [TX DMA] slot 3: 33554432 bytes in 495593 us (64 MB/s, 0.54 Gbps)
[  765.442941] axidma_user: [RX DMA] slot 3: 33554432 bytes in 18557 us (1724 MB/s, 14.46 Gbps)
[TX] sent chunk 28: slot=3 len=33554432 total=939524096 bytes
[RX] received chunk 21: slot=0 len=33554432 total=704643072 bytes
[  765.936962] axidma_user: [TX DMA] slot 0: 33554432 bytes in 494041 us (64 MB/s, 0.54 Gbps)
[  765.945257] axidma_user: [RX DMA] slot 0: 33554432 bytes in 18527 us (1727 MB/s, 14.48 Gbps)
[TX] sent chunk 29: slot=0 len=33554432 total=973078528 bytes
[RX] received chunk 22: slot=1 len=33554432 total=738197504 bytes
[  766.439120] axidma_user: [TX DMA] slot 1: 33554432 bytes in 493882 us (64 MB/s, 0.54 Gbps)
[  766.447422] axidma_user: [RX DMA] slot 1: 33554432 bytes in 18543 us (1725 MB/s, 14.47 Gbps)
[TX] sent chunk 30: slot=1 len=33554432 total=1006632960 bytes
[RX] received chunk 23: slot=2 len=33554432 total=771751936 bytes
[  766.941819] axidma_user: [TX DMA] slot 2: 33554432 bytes in 494419 us (64 MB/s, 0.54 Gbps)
[  766.950120] axidma_user: [RX DMA] slot 2: 33554432 bytes in 18500 us (1729 MB/s, 14.51 Gbps)
[TX] sent chunk 31: slot=2 len=33554432 total=1040187392 bytes
[RX] received chunk 24: slot=3 len=33554432 total=805306368 bytes
[  767.445413] axidma_user: [TX DMA] slot 3: 33554432 bytes in 495313 us (64 MB/s, 0.54 Gbps)
[  767.453717] axidma_user: [RX DMA] slot 3: 33554432 bytes in 18519 us (1727 MB/s, 14.49 Gbps)
[TX] sent chunk 32: slot=3 len=8388608 total=1048576000 bytes
[RX] received chunk 25: slot=0 len=33554432 total=838860800 bytes
[  767.946080] axidma_user: [TX DMA] slot 0: 33554432 bytes in 492384 us (64 MB/s, 0.54 Gbps)
[  767.954374] axidma_user: [RX DMA] slot 0: 33554432 bytes in 18485 us (1731 MB/s, 14.52 Gbps)
[RX] received chunk 26: slot=1 len=33554432 total=872415232 bytes
[  768.445723] axidma_user: [TX DMA] slot 1: 33554432 bytes in 491372 us (65 MB/s, 0.54 Gbps)
[  768.454016] axidma_user: [RX DMA] slot 1: 33554432 bytes in 18498 us (1729 MB/s, 14.51 Gbps)
[RX] received chunk 27: slot=2 len=33554432 total=905969664 bytes
[  768.945549] axidma_user: [TX DMA] slot 2: 33554432 bytes in 491553 us (65 MB/s, 0.54 Gbps)
[  768.953844] axidma_user: [RX DMA] slot 2: 33554432 bytes in 18496 us (1730 MB/s, 14.51 Gbps)
[RX] received chunk 28: slot=3 len=33554432 total=939524096 bytes[  769.437867] axidma_user: [TX DMA] slot 3: 8388608 bytes in 484045 us (16 MB/s, 0.13 Gbps)

[  769.449082] axidma_user: [RX DMA] slot 3: 8388608 bytes in 13777 us (580 MB/s, 4.87 Gbps)
[RX] received chunk 29: slot=0 len=33554432 total=973078528 bytes
[RX] received chunk 30: slot=1 len=33554432 total=1006632960 bytes
[RX] received chunk 31: slot=2 len=33554432 total=1040187392 bytes
[RX] received chunk 32: slot=3 len=8388608 total=1048576000 bytes
[main] transfer complete: 1048576000 bytes in 15.7589 s (63.46 MB/s, 0.53 Gbps)

root@zcu102axidma:~# diff test_1000mb_in.bin /tmp/test_1000mb_in.bin 

root@zcu102axidma:~# rm /tmp/test_1000mb_in.bin 

root@zcu102axidma:~# sync

Conclusion

Both the 3rd party driver dma-proxy and the custom driver axidma-user gives the average 60MB/s data rate at 200MHz without hardware cache coherency.

The actual performance with cache coherency is yet to be tested.

Also refer Zynq UltraScale+ MPSoC Cache Coherency

Linux, Xilinx, Petalinux, DMA
petalinux yocto zcu102 mpsoc fpga dma
This post is licensed under MIT by the author.

© - Sanjaybyranna. All rights reserved

Personal blog post

Last updated on October 22, 2026