首页 > 代码库 > [dpdk] 读文档
[dpdk] 读文档
前提：已读了这本书<<深入浅出dpdk（朱清河等著）>>。
目标：读官方文档，同时跟着文档进行安装编译等工作。
环境：thinkpadT450 + archlinux + kvm + CentOS 7（逐层自下而上的关系）
KVM参数：sudo qemu-system-x86_64 -nographic -enable-kvm -numa node -numa node -drive file=disk.img,if=virtio -cdrom /home/tong/ISO/CentOS-7-x86_64-DVD-1511.iso -boot order=c -m 2G -net nic,model=virtio,macaddr=‘00:00:00:00:00:03‘ -net tap,ifname=tap0
当前dpdk版本：dpdk-16.07
前提交代完，接下来是正文。内容会是一种笔记的形式，我遇见问题，就会在下边写下来。
一 编译：
make config T=x86_64-native-linuxapp-gccsed -ri ‘s,(PMD_PCAP=).*,\1y,‘ build/.configmake
问题一：编译错误
In file included from /root/dpdk-16.07/lib/librte_eal/linuxapp/eal/eal_pci.c:42:0:/root/dpdk-16.07/build/include/rte_memcpy.h:814:2: error: incompatible type for argument 2 of ‘_mm_storeu_si128’ MOVEUNALIGNED_LEFT47(dst, src, n, srcofs); ^In file included from /root/dpdk-16.07/build/include/rte_common.h:289:0, from /root/dpdk-16.07/build/include/rte_log.h:45, from /root/dpdk-16.07/lib/librte_eal/linuxapp/eal/eal_pci.c:37:/usr/lib/gcc/x86_64-redhat-linux/4.8.5/include/emmintrin.h:700:1: note: expected ‘__m128i’ but argument is of type ‘int’ _mm_storeu_si128 (__m128i *__P, __m128i __B)
咋解决：查看了这个头文件emmintrin.h 内容是这样的，也就是说这是个CPU平台相关的问题，所以很可能需要调整kvm的cpu参数，或者dpdk的编译参数，从这两个方面入手：
/* The Intel API is flexible enough that we must allow aliasing with other vector types, and their scalar components. */typedef long long __m128i __attribute__ ((__vector_size__ (16), __may_alias__));typedef double __m128d __attribute__ ((__vector_size__ (16), __may_alias__));
dpdk的编译参数并没有神马好改的，你重要指定一个CPU平台给它，所以我必须让qemu模拟intel或者是虚拟机直接认得本地CPU，然而qemu的手册并不清晰，又没有时间系统的学习qemu，只能试了。（qemu kvm也需要系统学习！）
尝试1：不给qemu CPU参数：
qemu-system-x86_64 -nographic -enable-kvm -m 2G -drive file=disk.img,if=virtio \
-net nic,model=virtio,macaddr=‘00:00:00:00:00:03‘ -net tap,ifname=tap0
进入系统后查看cpuinfo
[root@dpdk dpdk-16.07]# cat /proc/cpuinfo processor : 0vendor_id : GenuineIntelcpu family : 6model : 6model name : QEMU Virtual CPU version 2.5+stepping : 3......
编译报同样的错，毫无悬念。
尝试2：指定CPU类型，模拟至强E3
qemu-system-x86_64 -nographic -enable-kvm -m 2G -cpu IvyBridge -drive file=disk.img,if=virtio -net nic,model=virtio,macaddr=‘00:00:00:00:00:03‘ -net tap,ifname=tap0
编译成功：
[root@dpdk dpdk-16.07]# cat /proc/cpuinfo processor : 0vendor_id : GenuineIntelcpu family : 6model : 58model name : Intel Xeon E3-12xx v2 (Ivy Bridge)stepping : 9microcode : 0x1cpu MHz : 2394.456cache size : 4096 KB... ...
编译错误二：
SYMLINK-FILE include/exec-env/rte_dom0_common.h INSTALL-LIB librte_eal.a== Build lib/librte_eal/linuxapp/igb_uiomake: *** /lib/modules/3.10.0-327.el7.x86_64/build: No such file or directory. Stop.make[5]: *** [igb_uio.ko] Error 2make[4]: *** [igb_uio] Error 2make[3]: *** [linuxapp] Error 2make[2]: *** [librte_eal] Error 2make[1]: *** [lib] Error 2make: *** [all] Error 2
解决：人家文档写的清楚需要内核开发包 “kernel - devel.x86_64”
yum install kernel-develln -fs /usr/src/kernels/3.10.0-327.36.1.el7.x86_64/ /lib/modules/3.10.0-327.el7.x86_64/build
问题三：因为enable了PCAP，没错就是前边那个sed，鬼知道它是干嘛的，反正要装 libpcap-devel.
编译通过，好棒！: )
二，调整硬件架构
调整虚拟机为numa架构，并设置hugepage。
# numa参数只调整结构，并不申请资源，所以还是需要m和smp两个参数来进行资源申请。
sudo qemu-system-x86_64 -nographic -enable-kvm -m 2G -cpu Nehalem \ #Nehalem就是i7，我的物理CPU是i7，模拟至强的时候，warning有一些特性不支持，为了防止后续出现诡异的错误，就模拟了i7. -smp cores=2,threads=2,sockets=2 \ #两颗CPU，两个核心，超线程 -numa node,mem=1G,cpus=0-3,nodeid=0 -numa node,mem=1G,cpus=4-7,nodeid=1 -drive file=disk.img,if=virtio -net nic,model=virtio,macaddr=‘00:00:00:00:00:03‘ -net tap,ifname=tap0 &
然后在虚拟机里的效果就是这样的：
[root@dpdk ~]# cat /proc/cpuinfo processor : 0vendor_id : GenuineIntelcpu family : 6model : 26model name : Intel Core i7 9xx (Nehalem Class Core i7)stepping : 3microcode : 0x1cpu MHz : 2394.456cache size : 4096 KBphysical id : 0siblings : 4core id : 0cpu cores : 2apicid : 0initial apicid : 0fpu : yesfpu_exception : yescpuid level : 11wp : yesflags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx lm constant_tsc rep_good nopl pni ssse3 cx16 sse4_1 sse4_2 x2apic popcnt hypervisor lahf_lmbogomips : 4788.91clflush size : 64cache_alignment : 64address sizes : 40 bits physical, 48 bits virtualpower management:processor : 1vendor_id : GenuineIntelcpu family : 6model : 26model name : Intel Core i7 9xx (Nehalem Class Core i7)stepping : 3microcode : 0x1cpu MHz : 2394.456cache size : 4096 KBphysical id : 0siblings : 4core id : 0cpu cores : 2apicid : 1initial apicid : 1fpu : yesfpu_exception : yescpuid level : 11wp : yesflags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx lm constant_tsc rep_good nopl pni ssse3 cx16 sse4_1 sse4_2 x2apic popcnt hypervisor lahf_lmbogomips : 4788.91clflush size : 64cache_alignment : 64address sizes : 40 bits physical, 48 bits virtualpower management:processor : 2vendor_id : GenuineIntelcpu family : 6model : 26model name : Intel Core i7 9xx (Nehalem Class Core i7)stepping : 3microcode : 0x1cpu MHz : 2394.456cache size : 4096 KBphysical id : 0siblings : 4core id : 1cpu cores : 2apicid : 2initial apicid : 2fpu : yesfpu_exception : yescpuid level : 11wp : yesflags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx lm constant_tsc rep_good nopl pni ssse3 cx16 sse4_1 sse4_2 x2apic popcnt hypervisor lahf_lmbogomips : 4788.91clflush size : 64cache_alignment : 64address sizes : 40 bits physical, 48 bits virtualpower management:processor : 3vendor_id : GenuineIntelcpu family : 6model : 26model name : Intel Core i7 9xx (Nehalem Class Core i7)stepping : 3microcode : 0x1cpu MHz : 2394.456cache size : 4096 KBphysical id : 0siblings : 4core id : 1cpu cores : 2apicid : 3initial apicid : 3fpu : yesfpu_exception : yescpuid level : 11wp : yesflags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx lm constant_tsc rep_good nopl pni ssse3 cx16 sse4_1 sse4_2 x2apic popcnt hypervisor lahf_lmbogomips : 4788.91clflush size : 64cache_alignment : 64address sizes : 40 bits physical, 48 bits virtualpower management:processor : 4vendor_id : GenuineIntelcpu family : 6model : 26model name : Intel Core i7 9xx (Nehalem Class Core i7)stepping : 3microcode : 0x1cpu MHz : 2394.456cache size : 4096 KBphysical id : 1siblings : 4core id : 0cpu cores : 2apicid : 4initial apicid : 4fpu : yesfpu_exception : yescpuid level : 11wp : yesflags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx lm constant_tsc rep_good nopl pni ssse3 cx16 sse4_1 sse4_2 x2apic popcnt hypervisor lahf_lmbogomips : 4788.91clflush size : 64cache_alignment : 64address sizes : 40 bits physical, 48 bits virtualpower management:processor : 5vendor_id : GenuineIntelcpu family : 6model : 26model name : Intel Core i7 9xx (Nehalem Class Core i7)stepping : 3microcode : 0x1cpu MHz : 2394.456cache size : 4096 KBphysical id : 1siblings : 4core id : 0cpu cores : 2apicid : 5initial apicid : 5fpu : yesfpu_exception : yescpuid level : 11wp : yesflags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx lm constant_tsc rep_good nopl pni ssse3 cx16 sse4_1 sse4_2 x2apic popcnt hypervisor lahf_lmbogomips : 4788.91clflush size : 64cache_alignment : 64address sizes : 40 bits physical, 48 bits virtualpower management:processor : 6vendor_id : GenuineIntelcpu family : 6model : 26model name : Intel Core i7 9xx (Nehalem Class Core i7)stepping : 3microcode : 0x1cpu MHz : 2394.456cache size : 4096 KBphysical id : 1siblings : 4core id : 1cpu cores : 2apicid : 6initial apicid : 6fpu : yesfpu_exception : yescpuid level : 11wp : yesflags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx lm constant_tsc rep_good nopl pni ssse3 cx16 sse4_1 sse4_2 x2apic popcnt hypervisor lahf_lmbogomips : 4788.91clflush size : 64cache_alignment : 64address sizes : 40 bits physical, 48 bits virtualpower management:processor : 7vendor_id : GenuineIntelcpu family : 6model : 26model name : Intel Core i7 9xx (Nehalem Class Core i7)stepping : 3microcode : 0x1cpu MHz : 2394.456cache size : 4096 KBphysical id : 1siblings : 4core id : 1cpu cores : 2apicid : 7initial apicid : 7fpu : yesfpu_exception : yescpuid level : 11wp : yesflags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx lm constant_tsc rep_good nopl pni ssse3 cx16 sse4_1 sse4_2 x2apic popcnt hypervisor lahf_lmbogomips : 4788.91clflush size : 64cache_alignment : 64address sizes : 40 bits physical, 48 bits virtualpower management:
[root@dpdk ~]# numastat node0 node1numa_hit 193979 184758numa_miss 0 0numa_foreign 0 0interleave_hit 7016 6897local_node 187494 184103other_node 6485 655
大页：
查看cpuinfo： If pse exists, 2M hugepages are supported; if pdpe1gb exists, 1G hugepages are supported. 所以我只支持2M，而且我也没有那么多内存。但是我的物理cpu是支持1G的，这让我感觉很牛逼，我是说host主机。
通过设置内核参数：测试发现，保留的大页会平均分给两个node。
[root@dpdk ~]# cat /boot/grub2/grub.cfg|grep huge linux16 /vmlinuz-3.10.0-327.el7.x86_64 root=/dev/mapper/centos-root ro crashkernel=auto rd.lvm.lv=centos/root rd.lvm.lv=centos/swap rhgb quiet LANG=en_US.UTF-8 hugepages=32[root@dpdk ~]# cat /proc/meminfo |grep HugeAnonHugePages: 8192 kBHugePages_Total: 32HugePages_Free: 32HugePages_Rsvd: 0HugePages_Surp: 0Hugepagesize: 2048 kB[root@dpdk ~]# cat /sys/devices/system/node/node0/hugepages/hugepages-2048kB/nr_hugepages 16[root@dpdk ~]# cat /sys/devices/system/node/node1/hugepages/hugepages-2048kB/nr_hugepages 16
[dpdk] 读文档
声明：以上内容来自用户投稿及互联网公开渠道收集整理发布，本网站不拥有所有权，未作人工编辑处理，也不承担相关法律责任，若内容有误或涉及侵权可进行投诉： 投诉/举报 工作人员会在5个工作日内联系你，一经查实，本站将立刻删除涉嫌侵权内容。