Question:
I have read numerous documentations about the differences between coarse and fine grain parallelism, but I don't understand it very well, here is an example of what I have seen:
"An application shows fine-grained parallelism if its subtasks must communicate many times per second, it is considered coarse-grained parallelism if they do not communicate many times per second(…)" Source: Wikipedia.
When it comes to implementing, for example, a Matrix x Vector multiplication, how do a fine-grained implementation differ from a coarse-grained one?
I have already made a fine-grained solution by creating a thread for each row of the matrix and then operating with it, but if I now want to make a coarse-grained solution, how would I have to implement it?
What I have thought has been, in my case, to use the Subramanian equation with Coef. lock 0 for example to get the number of threads needed and then divide the dimension of the matrix by the number of threads to launch a thread per block and not by rows as in the fine-grained one.
Let's see if I can once and for all find out how each parallelism works.
Answer:
The type of parallelism you implement in a system is an architectural decision.
Imagine for example a system that calculates reflections, lights and shadows for a 3D object and then applies them to the basic view. Assuming that the three components can be computed independently of each other and there are 3 or more cores available for parallel computations, it makes sense to run 3 threads, one for reflections, one for highlights and one for shadows, applying them to the basic view after they are done. the parallel threads. That would be a coarse grain usage.
grano grueso
+--(reflejos)-+
/ \
/ \
---(main)---(crear hilos)+-----(luces)-------+(resultado)---(juntar capas)
\ /
\ /
+--(sombras)--+
In another example you want to implement a distributed system like a TCP chat server. Since the incoming messages are events that occur outside the control of the server's workflow and for technical reasons, the InputStream and OutputStream must be run in separate threads. The best function of the system is achieved if the distribution of messages works in tasks that are as short as possible. A paradigm such as a Scheduler could be used to broadcast messages on recyclable worker threads. That would be a use of fine thread.
grano fino
-(in){1,n}----(msg1)--+-(ent1)-(msg2)-(msg3)--+---+--
\
+-+(hilo_1)+--------------------
/ \
+----(hilo_2)---------------------
/ \
------(pool)+---+--(hilo_3)------------+--+------
\ \ \ / \
+++--(hilo_4)----+-------+--------
/ \ \ / \ / \ \
-(out)----(msg1)--+-(ent1)-(msg2)-(msg3)--+---+--