An easy explanation about how all the components of this pattern work together to concurrently process a batch of jobs.

Concurrent WorkerPool Pattern

Introduction


Synchronize worker executions by using Semaphore pattern instead of sync.WaitGroup.

Introduction


A theoretical and practical approach


A theoretical and practical approach

  1. The need to execute queries around your object becomes complex to deal with (multiple HTTP calls to other services, expensive joins across tables, etc).
  2. Performance degradation on your writes since your service ended up having more reading operation than writes.


A different approach to batch processing, and how to potentiate the power of data pipelines throughout the use of the Go concurrency model.

Introduction to pipelines

Previous problem and solution

Lucas Godoy

IT Project Lead at Mercadolibre

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store