Commits on Mar 25, 2015

1. Improve memory of producer retry queue …

   The old version just used a simple slice as a queue, using append and slice[1:].
   This had correct (amortized O(1)) running time, but did not behave well with the
   garbage collector. In particular:
   - elements removed from the slice were still pointed to by the underlying array,
     preventing them from being collected until the slice was resized
   - the slice would only be resized when its current capacity was "full", even if
     it had a substantial amount of free space it was wasting
   
   Switch instead to my queue implementation (which I wrote a while ago to solve
   these exact problems somewhere else). It uses a ringbuffer that is guaranteed to
   stay with a factor of 2 size of the actual number of messages stored, it
   generates a lot less garbage when the input and output rates are about the same,
   and it nils removed elements in the underlying array so the garbage collector
   can pick them up at its leisure.
   
   The only possible concern is that it uses `interface{}` but the usage is so
   restrained that type-safety is trivial and the performance will not be noticable
   given everything else the producer has to do.

   

   eapache committed Mar 25, 2015
   Configuration menu

   • View commit details
   • Copy full SHA for 9670282
   • Browse repository at this point

   Copy the full SHA

   9670282 View commit details

   Browse the repository at this point in the history