Re: [lwip-users] Why does enabling Debug fix this tcp_sndbuf()

[Osborne, David]

Damn you L1 cache!  I would disable you permanently if you weren’t so damn fast. :)

 

It appears cleaning the cache over the tx buffers address space has fixed the issue.  My only concern is, it shouldn’t have.  The tx buffers are set up as Normal Write through memory (T=0, C=1, B=0, S=0).  I have a sneaky suspicion cleaning the cache is simply providing the delay that the debug was (but hey a fix is a fix)

 

@Alister Fisher I am using your Ethernet patches.  I was hoping to switch back to the mainstream by now, but I’m not confident ST has fixed all the issues yet.

 

Thanks for your help everyone.

 

Cheers

David

 

From: lwip-users <lwip-users-bounces+david.osborne=leicabiosystems.com@nongnu.org> On Behalf Of Trampas Stern
Sent: Thursday, 25 March 2021 4:10 AM
To: Mailing list for lwIP users <lwip-users@nongnu.org>
Subject: Re: [lwip-users] Why does enabling Debug fix this tcp_sndbuf()

 

I had similar problem with an ATSAME70 which was an M7 core.  Turns out that the ethernet driver from Microchip did not invalidate cache for the ethernet MAC driver.  Hence it would work some of the time and not others, like when debugging was on.  Disabling the data cache on the processor would allow code to run with no errors. 

 

Microchip told me that none of their drivers were designed to work with cache on and to run ANY sample code or drivers required instruction and data cache to be turned off.   Needless to say I rewrote their drivers.

 

Also note the need for barrier instructions (__DSB(), __ISB())  these are often needed and vendor's drivers are missing them.  For example barriers are often needed at end of interrupt handlers.  Note even then some peripherals have caches in the peripherals that can not be flushed with barrier instructions, for example Flash memory.  

I also ran into this as that right after writing to flash memory I would read back the wrong values due to flash peripheral caching.  I had to create my own cache flush by reading large block of flash no near where I wrote, then returning to where I wrote and reading again.  The peripheral driver had a cache flush that did not work, so the reading different region was the only way to ensure the flash cache was flushed. 

 

 

The function below disables the cache in the core, part of core_cm7.h

 

 

/**
  \brief   Disable D-Cache
  \details Turns off D-Cache
  */
__STATIC_INLINE void SCB_DisableDCache (void)
{
  #if defined (__DCACHE_PRESENT) && (__DCACHE_PRESENT == 1U)
    register uint32_t ccsidr;
    register uint32_t sets;
    register uint32_t ways;

    SCB->CSSELR = 0U; /*(0U << 1U) | 0U;*/  /* Level 1 data cache */
    __DSB();

    SCB->CCR &= ~(uint32_t)SCB_CCR_DC_Msk;  /* disable D-Cache */
    __DSB();

    ccsidr = SCB->CCSIDR;

                                            /* clean & invalidate D-Cache */
    sets = (uint32_t)(CCSIDR_SETS(ccsidr));
    do {
      ways = (uint32_t)(CCSIDR_WAYS(ccsidr));
      do {
        SCB->DCCISW = (((sets << SCB_DCCISW_SET_Pos) & SCB_DCCISW_SET_Msk) |
                       ((ways << SCB_DCCISW_WAY_Pos) & SCB_DCCISW_WAY_Msk)  );
        #if defined ( __CC_ARM )
          __schedule_barrier();
        #endif
      } while (ways-- != 0U);
    } while(sets-- != 0U);

    __DSB();
    __ISB();
  #endif
}

 

 

On Wed, Mar 24, 2021 at 3:11 PM goldsimon@gmx.de <goldsimon@gmx.de> wrote:

Am 24.03.2021 um 16:32 schrieb Osborne, David:
> Hi All,
>
>  
>
> I’m using LwIP + FreeRTOS + Sockets on STM32H7 with external SDRAM and
> FLASH (Ethernet buffers are internal).  It works well until the server
> has to send a very large message (> 18Kbytes).  One in five attempts
> ends in failure.  During the transmission tcp_sndbuf() (in api_msg.c)
> returns zero and the connection falls over.  From what I’ve read, a
> common cause is that it’s waiting for an outstanding ACK.  I’m no
> expert, but it looks like the ACK has been sent (see Wireshark attachment).
>
>  
>
> When I enable certain debug settings it works every time (don’t you hate
> that).

That's an indication that there's something wrong that's hidden by
either longer runtime (more debug code enabled) or by heavier cache
usage (more code/data in the caches).

In any case, I'd suggest thoroughly review the OS port and the netif
driver regarding caching issues and/or threading issues (where interrupt
vs. thread/main loop also counts in).

Regards,
Simon

_______________________________________________
lwip-users mailing list
lwip-users@nongnu.org
https://lists.nongnu.org/mailman/listinfo/lwip-users

Please be advised that this email may contain confidential information. If you are not the intended recipient, please notify us by email by replying to the sender and delete this message. The sender disclaims that the content of this email constitutes an offer to enter into, or the acceptance of, any agreement; provided that the foregoing does not invalidate the binding effect of any digital or other electronic reproduction of a manual signature that is included in any attachment.