DEV Community

ouryperd
ouryperd

Posted on

Processing in parallel with Groovy

I had to place thousands of messages on a queue and collect each response by an ID. This is how I used multiprocessing to do that, so that messages were processed in parallel to make it complete faster:

//lists to keep the thread instances
List threads1 = []
List threads2 = []

//first task (e.g. putting on Q)
def doThis = { i, j ->
    sleep(randNum(100))
    println "doThis(${i}) thread: ${j}"
}

//second task (e.g. getting from Q)
def doThat = { i, j ->
    sleep(randNum(20000))
    println "doThat(${i}) thread: ${j}"
}

//call first task (e.g. for eachRow from DB, put msg on Q)
20.times { i ->
    def t1
    t1 = Thread.start { doThis(i, t1.name) }
    threads1.add(t1)
}

//wait for all task 1 threads to end
for (thread in threads1) {
    thread.join()
    println "doThis joined: ${thread.name}"
}

//call second task (e.g. for each correlation ID, get msg from Q)
20.times { i ->
    def t2
    t2 = Thread.start { doThat(i, t2.name) }
    threads2.add(t2)
}

//wait for all task 2 threads to end
for (thread in threads2) {
    thread.join()
    println "doThat joined: ${thread.name}"
}

println "All threads done"

def randNum(maxSize) { //create random wait times
    return Math.abs(new Random().nextInt() % maxSize) + 1 
}
Enter fullscreen mode Exit fullscreen mode

Top comments (1)

Collapse
 
djkianoosh profile image
Kianoosh Raika

take a look at the groovy gpars library