<html><head><meta http-equiv="Content-Type" content="text/html charset=windows-1252"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;"><br><div><div>On Dec 23, 2013, at 5:15 AM, Luo, Zheng <<a href="mailto:luozheng@wustl.edu">luozheng@wustl.edu</a>> wrote:</div><br class="Apple-interchange-newline"><blockquote type="cite">
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
<div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;">
<br>
<div>
<div>On Dec 23, 2013, at 2:14, Davide Compagnin <<a href="mailto:dcompagn@gmail.com">dcompagn@gmail.com</a>> wrote:</div>
<br class="Apple-interchange-newline">
<blockquote type="cite">
<div style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;"><p dir="ltr">Hi Luo, a possibility is to change the max number of processors in the kernel configuration file.</p>
<div class="gmail_quote">Il 22/dic/2013 18:59 "Luo, Zheng" <<a href="mailto:luozheng@wustl.edu">luozheng@wustl.edu</a>> ha scritto:<br type="attribution">
<blockquote class="gmail_quote" style="margin: 0px 0px 0px 0.8ex; border-left-width: 1px; border-left-color: rgb(204, 204, 204); border-left-style: solid; padding-left: 1ex;">
<div dir="ltr">
<div name="divtagdefaultwrapper" style="font-size: 12pt; margin: 0px; font-family: Calibri, Arial, Helvetica, sans-serif;">
<div><font color="#282828">Hi everyone,</font></div>
<div><font color="#282828"><br>
</font></div>
<div><font color="#282828">I want to use arbitrary number of cores to do run the global EDF. I tried several method, however none of them works for me. I think I truly need some help now.</font></div>
<div><br>
</div>
<div>1***</div>
<div><font color="#282828">First, I tried the C-EDF, I think I can group the cores that will be used in the experiment to a cluster and the unused cores to a cluster. However, I read the documentation on the website, it showed how to </font><font color="#282828" size="3">use
the L1,L2,L3 and ALL to do the division. I choose the parameter ALL, and run the experiment by choosing the ./setsched C_EDF. In the base_mt_task.c , I added:</font></div>
<div><font color="#282828" size="3"><br>
</font></div>
<div><font color="#282828" size="3">
<div>CALL( init_rt_thread() );</div>
<div>int cluster = 0;</div>
<div>int cluster_sz = 4; </div>
<div>be_migrate_to_cluster(cluster, cluster_sz);</div>
<div>param.cpu = cluster_to_first_cpu(cluster, cluster_sz);</div>
<div>CALL( set_rt_task_param(gettid(), ¶m) );</div>
</font></div>
<div><font size="3" color="#282828"><br>
</font></div>
<div><font size="3" color="#282828">I tried different number of the cluster and cluster_sz, when I used the different number, I did the method said in the website: </font><font face="Calibri, Arial, Helvetica, sans-serif" size="3"><tt>C-EDF</tt> plugin must
be reloaded (for example by switching to the <tt>Linux</tt> plugin and back to <tt>C-EDF). </tt></font><font size="3">When I see the cpu utilization by run top -1, all 8 cores are being used. I get very confused here. I don't know where I did wrong. I send
the screenshot as an attachment. I also send the base_mt_task.c as an attachment too.</font></div>
<div><br>
</div>
<div>2***</div>
<div><font size="3" color="#282828">Second, I tried to use the G-EDF by disabling the cores in the linux system. I used: </font></div>
<div>
<div>sudo echo "0" > /sys/devices/system/node/node0/cpu1/online</div>
<div>sudo echo "0" > /sys/devices/system/node/node0/cpu3/online</div>
</div>
<div><font color="#282828" size="3">Those command did disable the CPU cores, however the G-EDF seems not working correctly. The execution order of the rt_thread, seems not according to the G-EDF. Some of the late deadline thread, even finished early. I am really
confused here.</font></div>
<div><span style="color: rgb(40, 40, 40); font-size: 12px;"><br>
</span></div>
<div><span style="color: rgb(40, 40, 40); font-size: 12px;">3***</span></div>
<div><font color="#282828" size="3">Third, I used the Native Linux user-space command to set the number of cores, by using </font></div>
<div>
<div><font color="#282828" size="3">cpu_set_t mask;</font></div>
<div><font color="#282828" size="3">CPU_ZERO(&mask);</font></div>
<div><font color="#282828" size="3">for (unsigned i = first_core; i <= last_core; ++i) </font><span style="color: rgb(40, 40, 40); font-size: 12px;">{</span></div>
<div><font color="#282828" size="3"><span style="white-space: pre-wrap;"></span>CPU_SET(i, &mask);</font></div>
<div><font color="#282828" size="3">}</font></div>
<div><span style="color: rgb(40, 40, 40); font-size: 12px;">sched_setaffinity( 0 , sizeof(mask), &mask);</span></div>
</div>
<div><span style="color: rgb(40, 40, 40); font-size: 12px;">Then I run the experiment, and I will get same result as #1 method, it seems that G-EDF ignored the CPU mask setting.</span></div>
<div><span style="color: rgb(40, 40, 40); font-size: 12px;"><br>
</span></div>
<div><font color="#282828" size="3">I tired those three method, however none of them worked. Or is there any better idea to specify the number of online cores to run the experiment? </font><span style="font-size: 12pt; color: rgb(40, 40, 40);">I think I truly</span><span style="font-size: 12pt; color: rgb(40, 40, 40);"> need
some help now. Thank you very much. Looking forward to your reply.</span></div>
<div><font color="#282828"><br>
</font></div>
<div><font color="#282828"><br>
</font></div>
<div><font color="#282828">Zheng Luo</font></div>
</div>
</div>
<br>
_______________________________________________<br>
litmus-dev mailing list<br>
<a href="mailto:litmus-dev@lists.litmus-rt.org">litmus-dev@lists.litmus-rt.org</a><br>
<a href="https://lists.litmus-rt.org/listinfo/litmus-dev" target="_blank">https://lists.litmus-rt.org/listinfo/litmus-dev</a><br>
<br>
</blockquote>
</div>
_______________________________________________<br>
litmus-dev mailing list<br>
<a href="mailto:litmus-dev@lists.litmus-rt.org">litmus-dev@lists.litmus-rt.org</a><br>
<a href="https://lists.litmus-rt.org/listinfo/litmus-dev">https://lists.litmus-rt.org/listinfo/litmus-dev</a></div>
</blockquote>
<br>
</div>
<br>
<div><br>
</div>
<div>Hello Davide,</div>
<div><br>
</div>
<div>Thank you very much for the reply. That is one of the possibility. However, I think change the max number of processors in the kernel takes a lot of time. Because when I run the experiment, I will change the number of the cores very often. After change
the kernel, it takes some time to recompile it and reinstall it. Is there any other way of not recompile and reinstall the kernel ?</div>
<div><br>
</div>
<div><br>
</div>
<div>Zheng Luo</div>
</div></blockquote><br></div><div>Hi Zheng,</div><div><br></div><div>You can: (1) Compile the Linux kernel with NR_CPUS = to the number of system CPUs, and then (2) use the kernel boot parameter “<span style="white-space: pre-wrap;">maxcpus” and set any value less than or equal to NR_CPUS. You can specify kernel boot parameters via bootloader configuration (i.e., grub). There are many tutorials for this online.</span></div><div><span style="white-space: pre-wrap;"><br></span></div><div><span style="white-space: pre-wrap;">Of course, using kernel boot parameters will require you to reboot your machine every time you want to change the number of CPUs. The approach also has some other deficiencies:</span></div><div><span style="white-space: pre-wrap;">1) It’s hard to reason about how CPUs are connected to each other via cache.</span></div><div><span style="white-space: pre-wrap;">2) You may still only have one release master CPU.</span></div><div><span style="white-space: pre-wrap;"><br></span></div><div><span style="white-space: pre-wrap;">If any of the following statements are true, then you’re going to have to write a new Litmus scheduler plugin (or modify/extend an existing one):</span></div><div><span style="white-space: pre-wrap;">1) Need more than one non-real-time CPU (and you can’t accomplish this by having empty clusters in C-EDF).</span></div><div><span style="white-space: pre-wrap;">2) Can’t reboot between scheduler reconfigurations.</span></div><div><br></div><div>Going back to your original question, I think I may see some mistakes in your reasoning for item 1***. C-EDF with “ALL” is equivalent to G-EDF. C-EDF with L1 is equivalent to partitioned scheduling (on most systems when hyperthreading is disabled, at least). C-EDF with L2 means that clusters are created around shared L2 caches. For example, suppose you have CPUs 0 through 3. CPU 0 and CPU 1 share an L2. CPU 2 and CPU 3 share a distinct L2. C-EDF with L2 would create two clusters: {CPU0, CPU1} and {CPU2, CPU3}. You can extend this logic to clustering around L3 (this usually means grouping CPUs by CPU sockets—if your system has one socket, L3 clustering is probably equivalent to ALL). NOTE: On some systems, the L2 is also private to each CPU (e.g., every Nehalem and later CPU by Intel). In this case, L2 clustering would also be equivalent to partitioned scheduling. C-EDF on a single-socket post-Nehalem Intel platform is boring: Your configurations are equivalent to partitioned EDF or global EDF, with nothing interesting in-between.</div><div><br></div><div>Regarding liblitmus’s be_migrate_to_cluster(), I believe that I have found a bug on systems where there is more than one level of shared caches (the system may also need to be multi-socket). Such a CPU would be Intel’s older Dunnington Xeon, where pairs of CPUs share an L2 and groups of six CPUs share an L3. However, only the L3 is shared in Intel’s Nehalem and later CPUs, so I think you’re probably safe. If C-EDF doesn’t barf on you when a task enters real-time mode, I think you’re all right. Unfortunately, a good fix will be pretty involved—lots of cache probing is necessary—and I won’t have the time to develop a fix until after January. I would be overjoyed if someone took a crack at it themselves. Contact me if you’re interested.</div><br><div>-Glenn</div></body></html>