simple example of pci driver with dma

[Shaked Matzner]

Hey All,

So I'm trying to learn how pci devices and drivers work using the edu device and an educational driver, and It seems like the pci_dma_write function fails to actually write the information from the the dma buffer into the allocated address in the ram. Here is How I testsed it: 1.I've initialized the first byte of the dma buff to be 0x12 in the realize function of the edu:

 edu->dma_buf[0] = 0x12;

In the driver I've defined a struct save both the major and the destination address allocated through the dma cohherent:

 struct test {

      int major;

      void *vaddr_to;

    };

allocated it on the probe function and commanded the device to read 4 bytes:

 dev_info(&(dev->dev),"Hello from Shaked\n");

 dma_addr_t dma_handle_from, dma_handle_to;

 void *vaddr_from, *vaddr_to;

 enum { SIZE = 4 };

 /* device -> RAM. */

   vaddr_to = dma_alloc_coherent(&(dev->dev), 4, &dma_handle_to, GFP_ATOMIC);

   *((volatile int*)vaddr_to) = 0xff;

    test->vaddr_to = vaddr_to;

    dev_info(&(dev->dev), "vaddr_to = %px\n", vaddr_to);

    dev_info(&(dev->dev), "dma_handle_to = %llx\n", (unsigned long long)dma_handle_to);

    /*write source, dest, number of bytes to transfer and activate the dma timer)/*

    iowrite32(DMA_BASE, mmio + IO_DMA_SRC);

    iowrite32((u32)dma_handle_to, mmio + IO_DMA_DST);

    iowrite32(SIZE, mmio + IO_DMA_CNT);

    iowrite32(DMA_CMD | DMA_FROM_DEV | DMA_IRQ, mmio + IO_DMA_CMD);

I've changed a little bit the interrtupt and added a print to check the value I just read:

static irqreturn_t irq_handler(int irq, void *dev)

{

    int devi;

    irqreturn_t ret;

    u32 irq_status;

    struct test* test;

    test =  (struct test *)dev;

    if (test->major == major) {

        irq_status = ioread32(mmio + IO_IRQ_STATUS);

        pr_info("irq_handler irq = %d dev = %d irq_status = %llx\n",

                irq, devi, (unsigned long long)irq_status);

        /* Must do this ACK, or else the interrupts just keeps firing. */

        iowrite32(irq_status, mmio + IO_IRQ_ACK);

        pr_info("*vaddr_to new_value = %u\n", (*((u8*)test->vaddr_to)));

        ret = IRQ_HANDLED;

    } else {

        ret = IRQ_NONE;

    }

    return ret;

}

however the value I get is still 255(0xff) and not 18(0x12) probably I've missed something but when the interrupt is called the transfer to the RAM address should be completed, however it seems like the dma_write_buffer function from the device does not perform any transfer. What Am I missing?

when I debugged the dma_coherent_alloc I saw it uses direct acccess(without iommu), however when the dma_pci_write is called it tries to access the memory via iommu and get's an error code(MEMTX_DECODE_ERROR) from memory.c 's memory_region_dispatch_write. Since memory_region_access_valid returns false on it's first condition(also under memory.c

I am now in a deadlock, since even this simple example  doesn’t seem to work, any idea what happens, or any suggestions for an alternative simple device with a simple driver which I can learn from will be great.

Technical details: edu_device: https://github.com/qemu/qemu/blob/master/hw/misc/edu.c original edu_driver: https://github.com/cirosantilli/linux-kernel-module-cheat/blob/master/kernel_modules/qemu_edu.c I run this on an x86-64 qemu machine with the following configuration:

$repo_loc/build/qemu-system-x86_64 \

    -no-kvm \

    -kernel $repo_loc/linux-5.8.5/arch/x86/boot/bzImage \

    -boot c -m 2049M \

    -hda $repo_loc/buildroot-2020.02.6/output/images/rootfs.ext4 \

    -append "root=/dev/sda rw console=ttyS0,115200 acpi=off nokaslr" \

    -serial stdio -display none \

    -virtfs local,path=$repo_loc/shared,mount_tag=host0,security_model=passthrough,id=host0 \

    -device edu

(where $repo_location is the path to my build dir)

 

Thanks,

           Shaked Matzner

 

The contents of this email message and any attachments are intended solely for the addressee(s) and may contain confidential and/or privileged information and may be legally protected from disclosure. If you are not the intended recipient of this message or their agent, or if this message has been addressed to you in error, please immediately alert the sender by reply email and then delete this message and any attachments. If you are not the intended recipient, you are hereby notified that any use, dissemination, copying, or storage of this message or its attachments is strictly prohibited.