Re: [Qemu-devel] [PATCH v3 2/5] util: introduce threaded workqueue

[Xiao Guangrong]

On 11/26/18 6:56 PM, Dr. David Alan Gilbert wrote:
> * Xiao Guangrong (address@hidden) wrote:
> > On 11/23/18 7:02 PM, Dr. David Alan Gilbert wrote:
> >
> > > > +#include "qemu/osdep.h"
> > > > +#include "qemu/bitmap.h"
> > > > +#include "qemu/threaded-workqueue.h"
> > > > +
> > > > +#define SMP_CACHE_BYTES 64
> > >
> > > That's architecture dependent isn't it?
> >
> > Yes, it's arch dependent indeed.
> >
> > I just used 64 for simplification and i think it is <= 64 on most CPU arch-es
> > so that can work.
> >
> > Should i introduce statically defined CACHE LINE SIZE for all arch-es? :(
>
> I think it depends why you need it; but we shouldn't have a constant
> that is wrong, and we shouldn't define something architecture dependent
> in here.

I see. I will address Emilio's suggestion that rename SMP_CACHE_BYTES to
THREAD_QUEUE_ALIGN and additional comments.

> > > > +   /*
> > > > +     * the bit in these two bitmaps indicates the index of the ＠requests
> > > > +     * respectively. If it's the same, the corresponding request is free
> > > > +     * and owned by the user, i.e, where the user fills a request. Otherwise,
> > > > +     * it is valid and owned by the thread, i.e, where the thread fetches
> > > > +     * the request and write the result.
> > > > +     */
> > > > +
> > > > +    /* after the user fills the request, the bit is flipped. */
> > > > +    uint64_t request_fill_bitmap QEMU_ALIGNED(SMP_CACHE_BYTES);
> > > > +    /* after handles the request, the thread flips the bit. */
> > > > +    uint64_t request_done_bitmap QEMU_ALIGNED(SMP_CACHE_BYTES);
> > >
> > > Patchew complained about some type mismatches; I think those are because
> > > you're using the bitmap_* functions on these; those functions always
> > > operate on 'long' not on uint64_t - and on some platforms they're
> > > unfortunately not the same.
> >
> > I guess you were taking about this error:
> > ERROR: externs should be avoided in .c files
> > #233: FILE: util/threaded-workqueue.c:65:
> > +    uint64_t request_fill_bitmap QEMU_ALIGNED(SMP_CACHE_BYTES);
> >
> > The complained thing is "QEMU_ALIGNED(SMP_CACHE_BYTES)" as it gone
> > when the aligned thing is removed...
> >
> > The issue you pointed out can be avoid by using type-casting, like:
> > bitmap_xor(..., (void *)&thread->request_fill_bitmap)
> > cannot we?
>
> I thought the error was just due to long vs uint64_t ratehr than the
> qemu_aligned.  I don't think it's just a casting problem, since I don't
> think the long's are always 64bit.

Well, i made some adjustments that makes check_patch.sh really happy :),
as followings:
$ git diff util/
diff --git a/util/threaded-workqueue.c b/util/threaded-workqueue.c
index 2ab37cee8d..e34c65a8eb 100644
--- a/util/threaded-workqueue.c
+++ b/util/threaded-workqueue.c
@@ -62,21 +62,30 @@ struct ThreadLocal {
      */

     /* after the user fills the request, the bit is flipped. */
-    uint64_t request_fill_bitmap QEMU_ALIGNED(SMP_CACHE_BYTES);
+    struct {
+        uint64_t request_fill_bitmap;
+    } QEMU_ALIGNED(SMP_CACHE_BYTES);
+
     /* after handles the request, the thread flips the bit. */
-    uint64_t request_done_bitmap QEMU_ALIGNED(SMP_CACHE_BYTES);
+    struct {
+        uint64_t request_done_bitmap;
+    } QEMU_ALIGNED(SMP_CACHE_BYTES);

     /*
      * the event used to wake up the thread whenever a valid request has
      * been submitted
      */
-    QemuEvent request_valid_ev QEMU_ALIGNED(SMP_CACHE_BYTES);
+    struct {
+        QemuEvent request_valid_ev;
+    } QEMU_ALIGNED(SMP_CACHE_BYTES);

     /*
      * the event is notified whenever a request has been completed
      * (i.e, become free), which is used to wake up the user
      */
-    QemuEvent request_free_ev QEMU_ALIGNED(SMP_CACHE_BYTES);
+    struct {
+        QemuEvent request_free_ev;
+    } QEMU_ALIGNED(SMP_CACHE_BYTES);
 };
 typedef struct ThreadLocal ThreadLocal;

$ ./scripts/checkpatch.pl -f util/threaded-workqueue.c
total: 0 errors, 0 warnings, 472 lines checked

util/threaded-workqueue.c has no obvious style problems and is ready for 
submission.

check_patch.sh somehow treats QEMU_ALIGNED as a function before the 
modification.

And yes, u64 is not a long type on 32 bit arch, it's long[2] instead. that's 
fine
when we pass the &(u64) to the function whose parameter is (long *). I thing 
this
trick is widely used. e.g, the example in kvm that i replied to Emilio:

static inline bool kvm_test_request(int req, struct kvm_vcpu *vcpu)
{
    return test_bit(req & KVM_REQUEST_MASK, (void *)&vcpu->requests);
}