Drush executes a command or script in a single process. This means that performing long and heavy operations can lead to memory leaks due to the accumulation of static cache. This is especially noticeable when performing CRUD operations for a large number of entities, for example, when performing batch processing (batch). Drush has a mechanism for handling such situations in the context of batch operations. If more than 50% of the available memory is consumed, the process will be restarted. However, there are two problems in this case:

  1. The protection will only work when the batch is executed. If the butch is not used, then execution will continue until all available memory is used up. On the one hand, this means that data processing is implemented incorrectly and a butch should be used. On the other hand, we may be talking about already implemented commands, the time for refactoring of which has not yet been allocated. In addition, the code being run can be in a separate script and the butch is not used there, respectively.
  2. On some configurations, Drush cannot restart the process and processing is interrupted without reaching its completion. Ideally, this problem should be solved by the server settings, however, in the absence of such a possibility, the problem can be solved programmatically.

The software solution assumes timely cleaning of the static cache. When using a batch, cleaning should be performed on each operation or every few operations. If the butch is not used, the cleanup should be run at each iteration of the loop or every few iterations.

// Reset static cache when the operation is called from Drush.
// This avoids memory leaks in case of processing a large number of entities.
if (drupal_is_cli() && function_exists('drush_memory_limit')) {
  // Prevent already processed entities from being stored in memory.

  // Prevent starting a new thread (on some configurations it crashes the
  // whole process). Drush starts new thread after exceeding 50% of
  // available memory. Therefore, reset the entire static cache at ~33%.
  if (drush_memory_limit() > 0 && (memory_get_usage() * 3) >= drush_memory_limit()) {

The solution has been tested for Drupal 7 and Drush 8.

Add new comment