I am working on a small, standalone application that is responsible for processing a relatively large file (~1.5 million lines). The code should take each line of the file and add it to one of two redis sets. For the sample program demonstrating the issue, I simply add each line to a set.
The sample application has 4 methods demonstrating the different combinations of using reactive/non-reactive templates and parallel/non-parallel code. The only method that fails to fully execute is the reactive template + parallel method. When this method executes, the counter for how many lines have been read from the original flux shoots very high very fast, but the number of entries added to the set is very low and on my machine, stops adding around 30k. Eventually, I end up with the following stack trace:
Exception in thread "SimplePauseDetectorThread_0" java.lang.OutOfMemoryError: Java heap spaceException in thread "SimplePauseDetectorThread_0" java.lang.OutOfMemoryError: Java heap space at org.LatencyUtils.PauseDetector.notifyListeners(PauseDetector.java:37) at org.LatencyUtils.SimplePauseDetector$SimplePauseDetectorThread.run(SimplePauseDetector.java:144)Exception in thread "Thread-5" org.springframework.context.ApplicationContextException: Failed to unregister LiveBeansView MBean; nested exception is java.lang.OutOfMemoryError: Java heap space at org.springframework.context.support.LiveBeansView.unregisterApplicationContext(LiveBeansView.java:102) at org.springframework.context.support.AbstractApplicationContext.doClose(AbstractApplicationContext.java:1005) at org.springframework.context.support.AbstractApplicationContext$1.run(AbstractApplicationContext.java:945)Caused by: java.lang.OutOfMemoryError: Java heap space