Does your application architecture include Message Queues to feed work items to backend batch processing such as “Update Product Inventory” or “Send out Notification Emails”? Message Queues work well in these use cases as they decouple your system components and allow your backend processing to asynchronously process requests. This decoupling allows the queue to grow under heavy load allowing the background job to catch up when there is less load on the system resulting in an evenly distributed work load throughout the day.This is a big advantage over synchronous processing, but what happens if your background jobs can’t keep up with all incoming requests? This is a more complex problem than it might seem at first!For example, you could add more background processes, reduce the number of items flowing into the queue, optimize your background job implementation, and many other possibilities. Where and how should you start optimizing?This post follows an interesting problem we worked through with one of our customers.Their eCommerce web application was adding about 40 inventory item update messages per minute into a processing queue. Unfortunately only six messages per minute could actually be processed by the background job, which caused a huge backlog and lead to badly outdated inventory information in the web application and internal systems.Instead of adding additional background processing jobs they analyzed the performance of the background process itself, identifying the WebSphere XML parser as performance bottleneck. After replacing it with Apache Xalan, they increased performance by 1000%, and could now process at least 60 messages per minute! The following chart shows the throughput of two queues in their system and the jump in de-queuing actions when they switched to the alternative XML Parser at about 10:20AM:Continue reading ...
|
