<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Baurice nafack on Medium]]></title>
        <description><![CDATA[Stories by Baurice nafack on Medium]]></description>
        <link>https://medium.com/@bnafack?source=rss-cc0efb73c14f------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/proxy/1*TGH72Nnw24QL3iV9IOm4VA.png</url>
            <title>Stories by Baurice nafack on Medium</title>
            <link>https://medium.com/@bnafack?source=rss-cc0efb73c14f------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Fri, 20 Nov 2026 08:58:36 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@bnafack/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com" rel="hub"/>
        <item>
            <title><![CDATA[You should learn parallel programming]]></title>
            <link>https://medium.com/@bnafack/you-should-learn-parallel-programming-d338ed01812d?source=rss-cc0efb73c14f------2</link>
            <guid isPermaLink="false">https://medium.com/p/d338ed01812d</guid>
            <category><![CDATA[parallel-computing]]></category>
            <category><![CDATA[openmp]]></category>
            <category><![CDATA[multithread]]></category>
            <category><![CDATA[threads]]></category>
            <category><![CDATA[mpi]]></category>
            <dc:creator><![CDATA[Baurice nafack]]></dc:creator>
            <pubDate>Sun, 27 Feb 2022 20:58:44 GMT</pubDate>
            <atom:updated>2022-02-28T09:08:29.469Z</atom:updated>
            <content:encoded><![CDATA[<p>In today’s world, multicore processors are the norm, and single-core processors are nearly extinct. When it comes to performance or scalability in STEM, concurrency and parallelism are fundamental.</p><p><strong>1. Problem statement</strong></p><p>Most people buy computers with the most recent features or performance such as 8 cores, 16 threads, 4 GB , 8 to 16 GB of RAM, and so on ( I will use these characteristics as a framework), hoping that their simulations will run faster. Did you know that serial programs only use one of the eight cores? Did you know that a serial program uses only one thread ( I will explain it in more detail later) amount 16? <strong>When you run a program that could be parallelized on a single core, you are wasting resources.</strong></p><p><strong>2. What are a core and thread?</strong></p><p>A core is a CPU’s processor that could focus on one task at a time. Today, computers have more than two cores such that a core can work on one task, while another core works a different task, so the more cores a CPU has, the more efficient it is. The core is divided into virtual cores called <strong>threads</strong>. When a core contains multiple threads (see figure 1), this is referred to as <strong>multithreading or hyper-threading </strong>( for Intel processors). Each core in our previous example had two threads. So the total number of threads is 16. The more <strong>threads</strong> your CPU has, the better is its performance because they are the ones that make CPU multitasking.</p><figure><img alt="https://docs.faircom.com/doc/ctserver/CPUsandThreads.htm" src="https://cdn-images-1.medium.com/max/336/1*c4rgfTkfzTKtraJQf6igbA.png" /><figcaption>figure 1: A single CPU chip (1 “socket”) with 4 cores and 8 threads ref:(<a href="https://docs.faircom.com/doc/ctserver/CPUsandThreads.htm">https://docs.faircom.com/doc/ctserver/CPUsandThreads.htm</a>)</figcaption></figure><p>When running a serial program that is parallelizable, only one core amount n existent is used. Only one of the two threads inside the single-core that is running your program is used. In general, you are wasting resources.</p><p><strong>3. Why should you parallelize your code?</strong></p><p>For two primary reasons</p><ul><li><strong>Time-to solve solution</strong></li></ul><p>a) to solve the same problem in less time (see figure 2)</p><p>b) to solve a larger problem at the same time</p><ul><li><strong>Problem size( data size)</strong></li></ul><p>To solve a problem that could not be solved on a single computational unit’s memory addressable ( when having data with 1 TB for instance)</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/590/1*htzyBjZGYZCbFkJGfj4L9A.png" /><figcaption>figure 2: KD-tree building runtimes</figcaption></figure><p>I wrote a C++ program that generates a Kd-tree data structure with 10^n 2D random numbers generated (n =power of data generated see figure 2). The plot shows that the serial code (thread number =1) with n=8 takes 150 minutes to build the Kd-tree, whereas the parallel code with 10 threads takes about 14 minutes. For n=9, 10 threads took about 96 minutes to build the Kd-tree, while 48 threads took 17 minutes. <strong>As you can see, multithreading saves time, so learn how to parallelize your code.</strong></p><p><strong>4. How to parallelize your code?</strong></p><p>The starting point of parallel programming with C++: <a href="https://chryswoods.com/parallel_c++/part2.html">chryswoods.com | Part 2: Parallel Programming using Intel Threading Building Blocks</a></p><p>The starting point of parallel programming with python: <a href="https://chryswoods.com/parallel_python/index.html">chryswoods.com | Parallel Programming with Python</a></p><p>Sample code for parallel programming in c++: <a href="https://github.com/B23579/openmp">https://github.com/B23579/openmp</a></p><p><strong>References:</strong></p><p>1- <a href="https://hpc.llnl.gov/documentation/tutorials/introduction-parallel-computing-tutorial##Models">Introduction to Parallel Computing Tutorial | HPC @ LLNL</a></p><p>2- <a href="https://hpc-tutorials.llnl.gov/openmp/programming_model/">OpenMP Programming Model | LLNL HPC Tutorials</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=d338ed01812d" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Point-to-point communication: ring topology and MPI programming]]></title>
            <link>https://medium.com/@bnafack/point-to-point-communication-ring-topology-and-mpi-programming-1d63ee4b4233?source=rss-cc0efb73c14f------2</link>
            <guid isPermaLink="false">https://medium.com/p/1d63ee4b4233</guid>
            <category><![CDATA[ring-topogy-mpi]]></category>
            <category><![CDATA[mpi]]></category>
            <category><![CDATA[deadlock]]></category>
            <category><![CDATA[blocking-communication]]></category>
            <category><![CDATA[non-blocking]]></category>
            <dc:creator><![CDATA[Baurice nafack]]></dc:creator>
            <pubDate>Fri, 14 Jan 2022 21:40:56 GMT</pubDate>
            <atom:updated>2022-01-14T21:40:56.427Z</atom:updated>
            <content:encoded><![CDATA[<ol><li><strong>MPI programming</strong></li></ol><p>The whole point of MPI (Message Passing Interface) is to make processes communicate. Two types of communication exist point-to-point and collective. Point-to-point communication is divided into two operations: Send and Receive. Its most basic forms are called blocking communications.<br>The process of sending a message will wait until the process receiving has finished receiving all the information before it can continue its work. If not, the sending process will be in a deadlock. Some applications may be severely limited by this. Non-blocking communication is an alternative. Instead of waiting for the send (or they receive) to complete, the process will continue working, checking intermittently whether the communication has been completed.</p><p>As part of this project, we will write two MPI programs that are based on blocking and non-communication in order to study the performance and scalability of a simple ring topology of one-dimensional processors with p processors, where each processor has a left and right neighbor.</p><ol><li><strong>1 Point-to-point communication: ring topology</strong></li></ol><p><strong>Description of the problem:</strong></p><p>the program should implement a stream of messages in both directions:</p><ul><li>As the first step, P sends a message ( msgleft=rank) to its left neighbor ( P-1) and received it from its right neighbor ( P+1), and sends another message( msgright= -right) to P+1 and received from P-1.</li><li>It them do enough iterations till all the processors receive back the initial message. At each iteration, each processor and its rank to receive messages originating from certain, processor P should have a tag proportional to its rank ( i.e tag = P*10). Provide the code that should print on a file the following output :<br>I am process irank and I have received np massages. My final message has tag and value msg-left, msg-right .</li></ul><p>where irank,np,itag and msg are parameters that should be printed by the program. Make sure that your code produces the correct answer, independent of the number of processes you use. Take the time of the program using MPI_walltime routines and produce a plot of the runtime as a function of P. If the total time is very small, you might need to repeat several times the iterations in order to get reasonable measurements.</p><ol><li><strong>2 Sending messages in blocking communication</strong></li></ol><p>A send operation sends a buffer of data of a certain type to another process with the following properties :</p><ul><li>A reference to a buffer: the reference is a pointer to a buffer. This array will hold the data that you wish to send from the current process to another.</li><li>A datatype: the datatype must correspond precisely to the data stored in the buffer.</li><li>A number of elements: this is the number of elements in the buffer that we want to send to the destination.</li><li>A tag: The tag is a simple integer that identifies the type of communication. This is a completely informal value that we put randomly.</li><li>A destination id: The rank of the process we want to send the data or message to.</li><li>A communicator: The communicator on which to send the data to.</li></ul><ol><li><strong>3 Receiving messages in blocking communication</strong></li></ol><p>Messages are received in exactly the same way as they are sent. Instead of a destination id, we will need a source id: the identifier of the process from which we are waiting for a message. On top of that, depending on if we are using blocking or non-blocking communications, we will need additional arguments.</p><ol><li><strong>4 Non-Blocking send and receive</strong></li></ol><p>The send and receive functions will return immediately, giving us more control of the flow of the program. The sending or receiving buffers cannot be modified after they are called, but the program is free to continue with other operations. MPI_Wait and MPI_Test must be used when the buffers need to be filled with data.<br>There is a new parameter here, a request. This is used to track each separate transfer made by the program. It can be used to check the status of a transfer using the MPI_Test function or to wait until it is complete by calling MPI_Wait.</p><ol><li><strong>5 MPI Point-to-Point: Deadlock</strong></li></ol><p>Deadlock is a common problem in parallel processing. This occurs when two or more processes are competing for the same set of resources. A typical scenario in communication involves two processes wishing to exchange messages: each is trying to send a message to the other, but neither is ready to accept a message, figure [ 1]. We wrote our ring program by considering the possibility of deadlock. To protect our program from deadlock, we use different call ordering between tasks. This entailed having one task post its receive first and the other post its send first. We also used a non-blocking send in order to solve this problem.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/445/1*LSVqI4XiddQbbSp2Dk-drQ.png" /><figcaption>figure 1. Deadlock illustration. We can see that they both call standard blocking sends at the same point in the program, but the receiver that corresponds to each task’s send occurs later in the other task’s program.</figcaption></figure><p>2. <strong>Experimental result: Model network performance</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/578/1*Uhw9brX_O-bjjNj0i0tkdg.png" /><figcaption>Figure 2: Ring topology Illustration</figcaption></figure><p>To investigate model network performance, we used 1500 and 2500 iterations to measure run times as a function of processor count. An iteration ends when each process receives its initial message. We use MPI_Wtime() to measure each processor’s run time for a single iteration, then record processor means time. We repeated the operation as many times as possible for P processors because, according to the law of large numbers, the average of the results obtained from a large number of trials should be close to the expected value and will tend to become closer as more trials are performed.</p><p>3.<strong> Result</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/587/1*QdVahlQ0vNBbmMzRBBhHnw.png" /><figcaption>Figure 3. Mean execution times for non-blocking communication with 1500 iteration.</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/587/1*ZeMZZ-aFOoCIdsS33D6EOg.png" /><figcaption>Figure 4: Mean execution times for blocking communication with 1500 iteration.</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/587/1*YP5TXPanBHHMS3LXLhHmGQ.png" /><figcaption>Figure 5: Mean execution times for non-blocking communication with 2500 iteration.</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/587/1*cAfGgL0x85RKV59eG9xf5w.png" /><figcaption>Figure 6: Mean execution times for non-blocking communication with 2500 iteration.</figcaption></figure><p>4. <strong>Result Interpretation</strong></p><p>In non-blocking communication, the running times started to increase quadratically when more than 22 processors were used. As the number of processors increases, the running times for blocking communication also increase.</p><p>5. <strong>MPI code</strong></p><p>Our MPI programs were written in c++ and run on an HPC cluster which provides us with 24 processors on a single node. The code and process to run all the programs I provided for ring topology is on the following git repository: <a href="https://github.com/B23579/ring">https://github.com/B23579/ring</a></p><p><strong>6. Perspective</strong></p><p>In this study, processors could be bound by or on the same nodes, by socket, or on the same socket, and then the results could be compared. Additionally, we can try to understand the significance of the outlier observed on the graph.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=1d63ee4b4233" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>