I normally advice people to use mnesia:activity(Type, F) rather than mnesia:transaction(F), and to always start with real transactions, then measure and – only if really necessary (and safe!), switch to dirty where neeeded. This can then be done by just changing Type from ‘transaction’ to ‘async_dirty’.

In my experience, the “iterate in small batches” should be one of the first points. It is very good advice. Also, monitor ets and mnesia tables to see if they keep growing. Inserting temporary objects and forgetting to delete them is a fairly common source of memory growth.

In other cases, a form of load control may well be what’s needed, making sure that the system doesn’t take on more work than it can handle (easy to do in an asynchronous environment). One very simple such device would be a gen_server that workers ask (synchronously) for permission before starting a new task. The server can monitor the ‘run_queue’ to guard against cpu overload, memory usage, number of running processes, etc., depending on where your bottlenecks are. Keep it very simple.