Re: [PATCH] concurrent block allocation for ext2 against 2.5.64

William Lee Irwin III (wli@holomorphy.com)
Sat, 15 Mar 2003 00:24:31 -0800

On Fri, Mar 14, 2003 at 10:44:13PM -0800, Andrew Morton wrote:
>> Nope. What we're trying to measure here is pure in-memory lock contention,
>> locked bus traffic, context switches, etc, etc. To do that we need to get
>> the IO system out of the picture.
>> One way to do that is to increase /proc/sys/vm/dirty_ratio and
>> dirty_background_ratio to 70% or so. You can still hit IO wait if someone
>> tries to truncate a file which pdflush is writing out, so increase
>> dirty_expire_centisecs and dirty_writeback_centisecs to 1000000000 or so...
>> Then, on the second run, when all the required metadata blocks are in
>> pagecache you should be able to get an IO-free run.
>
On Fri, Mar 14, 2003 at 11:05:11PM -0800, William Lee Irwin III wrote:
> Oh, sorry, I did increase dirty_ratio and dirty_background_ratio to 99,
> I forgot about dirty_writeback_centisecs though, I'll re-run with that.

Next pass involves lockmeter:

$ cat /proc/sys/vm/dirty_expire_centisecs
360000
$ cd /test/wli
$ ls
$ (time dbench 128) |& tee -a ~/dbench.output.log.7
zsh: correct '~/dbench.output.log.7' to '~/dbench.output.log.6' [nyae]? n
128 clients started
0 62477 206.65 MB/sec
Throughput 206.651 MB/sec 128 procs
dbench 128 143.50s user 3258.66s system 1574% cpu 3:36.04 total

vma samples %-age symbol name
c0106ff4 7617343 29.5286 default_idle
c01dc3b0 5212934 20.2079 __copy_to_user_ll
c01dc418 1806434 7.00263 __copy_from_user_ll
c0264bd0 1595815 6.18617 sync_buffer
c0108140 712115 2.76051 .text.lock.semaphore
c0119058 621494 2.40922 try_to_wake_up
c011a1bc 409622 1.5879 schedule
c0107d0c 278704 1.08039 __down
c011c4ff 263802 1.02263 .text.lock.sched
c0152ab0 260394 1.00942 __find_get_block_slow
c011fadc 247423 0.959134 profile_hook
c0264da0 231721 0.898265 add_event_entry
c0168aac 223276 0.865528 d_lookup
c01dc510 212968 0.825569 atomic_dec_and_lock
c0119dac 208443 0.808028 scheduler_tick
c015f3dc 192765 0.747253 path_lookup
c01522a0 191853 0.743717 file_move
c0119860 188927 0.732375 load_balance
c0122930 168016 0.651313 current_kernel_time
c010f6b8 166633 0.645952 timer_interrupt
c013ece4 160376 0.621697 check_highmem_ptes
c0133118 155858 0.604183 find_get_page

procs -----------memory---------- -----io---- --system-- ----cpu----
r b free buff cache bi bo in cs us sy id wa
0 0 48309664 568288 23360 0 0 1026 18 0 0 100 0
0 0 48274208 568288 23392 0 0 1028 256 0 1 99 0
91 0 48192928 569056 58272 0 0 1036 1790 1 20 79 0
49 0 47769920 580224 439808 0 0 1024 5071 2 94 4 0
70 0 47437248 582720 773952 0 0 1030 2366 1 95 5 0
56 1 47140576 586656 1062272 0 0 1022 1931 1 97 3 0
86 0 46901920 581568 1307168 0 0 1025 1800 0 98 2 0
55 0 46619744 584672 1585568 0 0 1028 2584 1 93 5 0
26 1 46343712 577600 1867360 0 0 1025 3387 1 87 12 1
20 0 46033184 575936 2176384 0 0 1028 5742 2 67 30 1
27 1 45834912 577056 2366304 0 0 1027 6072 2 66 31 1
24 1 45692128 577184 2504960 16 0 1029 8056 2 56 41 1
procs -----------memory---------- -----io---- --system-- ----cpu----
r b free buff cache bi bo in cs us sy id wa
18 0 45533280 581376 2659648 32 0 1037 8107 2 57 40 1
23 1 45386464 575264 2810592 32 0 1036 8227 2 54 42 2
4 4 45342240 577056 2852768 360 0 1112 3217 1 14 81 5
11 1 45283040 576160 2911712 140 0 1065 9003 1 26 71 2
8 1 45221856 580064 2968128 64 0 1043 11620 2 31 66 1
15 0 45181984 577024 3009120 92 0 1050 9451 2 40 57 1
10 0 45160224 579136 3027584 32 0 1038 9296 2 47 51 0
12 2 45198176 575200 2991840 4 1156 1205 9604 3 47 47 3
19 0 45284320 571872 2909440 0 72 1180 10532 2 42 54 1
21 0 45364064 574112 2827136 0 0 1028 9547 2 49 49 0
23 0 45368224 576384 2821504 0 0 1026 7902 3 57 41 0
22 0 45354720 571776 2839616 0 512 1127 7965 3 60 37 0
16 0 45315168 575072 2874272 0 0 1054 8367 3 59 39 0
19 0 45290592 578272 2898048 68 0 1043 8835 2 45 52 0
18 0 45244768 574432 2947424 8 640 1188 9696 2 45 51 2
18 0 45217184 576064 2973184 8 0 1028 10751 2 43 55 0
17 0 45192992 578080 2995040 24 0 1031 10860 2 40 58 0
21 0 45189408 572736 3004128 0 708 1201 8661 3 56 41 0
17 0 45219744 574112 2971392 8 0 1028 9531 3 46 51 0
26 0 45270176 575232 2919456 0 0 1028 9925 3 44 53 0
28 0 45332448 576672 2855968 0 0 1026 9073 3 50 47 0
procs -----------memory---------- -----io---- --system-- ----cpu----
r b free buff cache bi bo in cs us sy id wa
17 0 45365536 578496 2821440 4 0 1028 8403 3 53 44 0
14 0 45353056 572768 2840352 0 592 1173 7842 3 60 37 0
14 0 45307808 574400 2882176 0 0 1028 8073 3 60 38 0
20 0 45287776 576320 2902208 0 0 1028 8750 3 54 43 0
17 0 45245280 578240 2943424 4 0 1025 10229 2 44 54 0
19 0 45229408 571840 2965536 0 588 1173 9809 2 44 53 0
15 1 45210848 573664 2981216 12 0 1028 9992 2 46 52 0
18 0 45189024 574880 3002016 48 0 1040 8482 2 50 48 0
9 0 45216032 576032 2972800 0 0 1026 8795 3 53 44 0
12 0 45263968 577056 2923680 0 0 1027 10132 3 43 55 0
18 0 45312992 578080 2874432 0 0 1027 8791 3 51 46 0
20 1 45362336 575168 2828000 0 896 1118 8793 2 52 45 1
29 0 45345376 573760 2847296 0 76 1172 8315 3 56 41 0
30 0 45319840 575648 2871040 0 0 1032 7647 3 61 36 0
6 1 45289440 576864 2898432 12 0 1026 8163 2 53 44 0
16 0 45255520 578240 2932256 12 0 1030 9372 2 44 54 0
16 0 45228000 572800 2965376 0 1016 1227 10052 2 44 52 1
15 0 45216736 573920 2974880 0 0 1080 10110 2 46 51 0
21 0 45200672 574912 2990528 0 0 1025 8989 2 54 44 0
24 0 45214304 575776 2974880 0 0 1027 9274 3 50 47 0
20 0 45263712 576608 2923520 0 0 1026 8689 3 50 48 0
procs -----------memory---------- -----io---- --system-- ----cpu----
r b free buff cache bi bo in cs us sy id wa
22 0 45320800 577440 2867328 0 0 1027 9487 2 47 50 0
19 0 45344416 571488 2849888 0 632 1079 8074 3 58 39 0
23 0 45344096 573184 2848384 0 0 1132 8043 3 57 40 0
25 0 45324960 574464 2866784 0 0 1026 7457 3 64 33 0
17 0 45296992 575808 2893440 0 0 1027 9124 3 51 46 0
19 0 45265952 576992 2923648 0 0 1025 9579 2 50 48 0
23 0 45237280 577728 2949344 0 0 1030 10051 2 48 50 0
15 0 45208800 578912 2978656 0 0 1026 10072 2 47 51 0
19 0 45211552 568832 2985696 0 176 1070 8537 3 55 42 0
20 0 45215328 569792 2980736 0 0 1027 9434 3 47 50 0
12 0 45245408 570432 2949472 0 0 1026 9018 3 51 46 0
19 0 45317600 570816 2875904 0 0 1025 9767 3 47 50 0
13 0 45351712 571616 2842272 0 0 1030 9012 2 51 46 0
23 0 45342752 572640 2851232 0 0 1027 8134 3 59 38 0
23 0 45332832 573536 2859616 0 0 1026 8081 3 57 40 0
15 0 45304992 574816 2886560 0 0 1024 8929 2 54 44 0
19 0 45270752 575712 2919776 0 0 1027 9503 2 49 49 0
16 0 45230560 576896 2958208 0 0 1031 10190 2 44 54 0
22 0 45205344 577664 2981312 0 0 1024 9903 2 45 53 0
16 0 45206688 578720 2981024 4 0 1028 8860 2 53 44 0
20 0 45218080 572128 2976416 0 988 1168 9526 3 50 47 1
procs -----------memory---------- -----io---- --system-- ----cpu----
r b free buff cache bi bo in cs us sy id wa
20 0 45242400 572640 2950144 0 0 1133 8446 3 52 45 0
21 0 45312608 573024 2879936 0 0 1026 9444 3 48 49 0
22 0 45347104 573664 2844576 0 0 1026 8102 3 57 40 0
15 0 45344352 574368 2847584 0 0 1027 8310 3 57 40 0
27 0 45334304 575008 2856736 0 0 1026 8763 3 52 45 0
26 0 45303328 575712 2887648 0 0 1026 9057 2 53 44 0
17 0 45269984 576704 2920704 0 0 1026 9975 2 45 52 0
20 0 45222880 578240 2966080 0 0 1027 10239 2 44 53 0
23 0 45209760 573120 2983168 4 852 1138 10067 2 45 52 1
25 0 45197792 573856 2995200 0 0 1136 8914 3 51 46 0
17 0 45216288 574208 2974976 0 0 1017 9714 2 47 51 0
15 0 45255712 574720 2935488 0 0 1026 9560 3 48 49 0
20 0 45311072 575360 2879296 0 0 1027 8887 3 55 42 0
29 0 45339360 575904 2851072 0 0 1031 7836 3 61 36 0
24 0 45341280 576480 2849056 0 0 1022 7930 3 57 40 0
27 0 45330976 577504 2858400 0 0 1025 7902 3 60 37 0
25 0 45311264 578112 2877728 0 0 1027 8213 3 56 42 0
17 0 45271712 579264 2916256 20 0 1032 9453 2 47 51 0
21 0 45236064 573632 2957344 0 864 1240 10165 2 45 52 1
21 0 45214752 574336 2977152 0 0 1026 10164 2 43 54 0
28 0 45212512 574976 2979936 0 0 1026 9132 3 52 45 0
procs -----------memory--------- -----io---- --system-- ----cpu----
r b free buff cache bi bo in cs us sy id wa
22 0 45204512 575360 2985536 0 0 1028 9087 3 51 46 0
16 0 45234976 576096 2954816 0 0 1028 9246 3 52 46 0
28 0 45284640 576608 2903360 0 0 1028 8713 3 55 43 0
30 0 45327328 577120 2860992 0 0 1026 8932 3 53 44 0
23 0 45342048 577504 2847104 0 0 1026 8421 3 55 43 0
26 0 45334656 578048 2854144 0 0 1025 7808 3 56 41 0
26 0 45315616 578720 2872352 0 0 1027 8527 3 53 45 0
18 1 45280800 574528 2911712 0 524 1028 8218 2 55 42 0
22 0 45246752 573792 2946784 0 416 1256 9411 3 45 51 1
11 0 45226464 574432 2965856 0 0 1026 9924 2 44 54 0
13 0 45214432 575296 2977056 0 0 1027 9667 2 47 51 0
24 0 45214048 575872 2976544 0 0 1026 8562 2 53 45 0
26 0 45216544 576288 2973792 0 0 1026 8803 3 48 49 0
28 0 45261472 576640 2928544 0 0 1043 9363 3 50 47 0
27 0 45304352 576928 2884896 0 0 1009 8179 3 52 45 0
27 0 45349408 577472 2839488 0 0 1028 8289 3 55 42 0
17 0 45342432 577792 2845600 0 0 1033 8466 3 56 41 0
35 0 45328672 578112 2858560 0 0 1022 8160 2 56 41 0
23 0 45290720 578464 2897344 0 0 1026 8669 2 51 47 0
17 0 45249824 579360 2938080 0 0 1027 8979 2 51 47 0
14 0 45234080 572896 2960544 0 784 1215 10500 2 43 54 1
procs -----------memory---------- -----io---- --system-- ----cpu----
r b free buff cache bi bo in cs us sy id wa
20 0 45218656 573344 2974624 0 0 1030 10204 2 44 54 0
35 0 45216480 573664 2976992 0 0 1049 8737 2 51 46 0
15 0 45230048 574048 2962240 0 0 1277 10935 3 52 45 0
19 0 45256928 574432 2934816 0 0 1012 8437 3 50 47 0
18 0 45306080 574720 2885344 0 0 1025 8968 3 51 46 0
11 0 45346016 574944 2846176 0 0 1026 8805 2 53 45 0
21 0 45350688 575200 2842144 0 0 1026 7943 3 57 40 0
13 0 45315552 575520 2876352 0 0 1026 8419 2 56 42 0
24 0 45292704 575872 2897824 0 0 1028 8356 2 55 43 0
23 0 45254176 576576 2936992 0 0 1029 9594 3 49 49 0
18 0 45229856 577056 2960640 0 0 1027 9921 2 46 52 0
17 0 45209632 577600 2980864 0 0 1025 9635 2 45 53 0
26 0 45221664 577888 2967232 0 0 1045 9195 2 51 47 0
16 0 45231264 578464 2957248 0 0 1264 10989 2 51 47 0
21 0 45253088 578784 2935552 0 0 1025 8312 3 55 43 0
22 0 45310240 579040 2876992 0 0 1028 8878 3 51 47 0
23 0 45346016 579392 2841696 0 0 1026 9040 3 49 48 0
19 0 45345696 579712 2842464 0 0 1026 8184 2 58 39 0
23 0 45324000 580064 2863296 0 0 1027 8464 3 53 45 0
21 0 45285024 574240 2909024 0 748 1157 8943 2 52 45 0
17 0 45251360 574848 2941888 0 0 1081 9679 3 48 50 0
procs -----------memory---------- -----io---- --system-- ----cpu----
r b free buff cache bi bo in cs us sy id wa
16 0 45233760 575296 2959616 0 0 1027 10785 2 43 55 0
21 0 45208032 575904 2983936 20 0 1034 9635 2 46 52 0
11 0 45212512 576576 2977248 12 0 1030 9593 2 48 49 0
13 0 45226848 576832 2963360 0 0 1026 9295 3 47 51 0
27 0 45264544 577088 2924704 0 0 1026 8856 3 51 46 0
17 0 45308768 577280 2880608 0 0 1027 9160 3 51 46 0
28 0 45335584 577536 2854528 0 0 1030 8855 3 52 46 0
25 0 45338336 577856 2850976 0 0 1027 7700 3 57 40 0
29 0 45326816 578048 2862496 0 0 1025 7957 3 56 42 0
33 0 45291168 578400 2897952 0 0 1028 8537 2 55 43 0
17 0 45252576 578880 2936192 0 0 1024 8634 2 53 44 0
17 0 45226912 579136 2960416 0 0 1026 9436 2 49 49 0
15 0 45206240 579392 2980928 0 0 1028 9701 2 51 47 0
26 0 45210976 579520 2978112 0 0 1026 9054 3 52 45 0
28 0 45227040 579712 2960576 0 0 1025 9191 3 47 50 0
25 0 45255328 579968 2932736 0 0 1027 8116 3 55 42 0
20 0 45301152 580096 2886624 0 0 1027 9113 3 52 45 0
23 0 45324128 572960 2870272 0 612 1129 7719 3 57 40 0
22 0 45341792 573216 2852512 0 0 1075 8123 3 54 43 0
23 0 45341728 573760 2851616 0 0 1027 8674 2 55 43 0
18 0 45306720 573984 2887360 0 0 1026 8670 2 55 43 0
procs -----------memory---------- -----io---- --system-- ----cpu----
r b free buff cache bi bo in cs us sy id wa
16 0 45282016 574176 2911232 0 0 1028 9431 3 50 48 0
17 0 45243040 574432 2951456 0 0 1030 9618 3 49 48 0
21 0 45218784 574656 2974240 0 0 1025 9529 2 49 49 0
21 0 45219296 574848 2971904 0 0 1027 9826 2 48 49 0
21 0 45234464 575008 2956288 0 0 1026 9718 3 48 50 0
17 0 45265376 575168 2927712 0 0 1028 8749 3 57 40 0
22 0 45278752 575232 2914176 0 0 1026 8613 3 55 42 0
31 0 45304480 575520 2887328 0 0 1025 7851 3 60 37 0
20 0 45329888 575808 2861280 0 0 1026 8646 3 54 43 0
24 0 45331872 575936 2859552 0 0 1027 8722 2 54 44 0
21 0 45305056 576192 2886976 0 0 1026 8383 2 58 40 0
24 0 45279136 576416 2912736 0 0 1027 8586 2 57 41 0
21 0 45248736 576896 2942272 0 0 1026 9470 2 51 47 0
14 0 45223392 577184 2967328 0 0 1026 10033 2 47 50 0
19 0 45219104 577376 2971392 0 0 1026 9856 2 49 49 0
17 0 45222432 577600 2967456 0 0 1027 9484 3 47 51 0
29 0 45261472 577632 2927264 0 0 1035 8997 3 54 43 0
25 0 45271008 577792 2917664 0 0 1021 8259 3 55 42 0
24 0 45291936 578016 2897632 0 0 1027 7985 3 57 40 0
22 0 45308960 578336 2880864 0 0 1027 9218 3 52 46 0
18 0 45328160 578560 2861120 0 0 1025 8907 2 51 47 0
procs -----------memory---------- -----io---- --system-- ----cpu----
r b free buff cache bi bo in cs us sy id wa
22 0 45304480 578784 2884768 0 0 1026 8257 2 52 45 0
36 0 45279776 578912 2909472 0 0 1027 8651 2 54 43 0
28 0 45254624 579104 2933280 0 0 1026 9417 3 49 48 0
22 0 45238688 579328 2949920 0 0 1031 8350 2 57 41 0
20 0 45216800 579552 2972384 0 0 1023 8923 2 51 47 0
22 0 45217504 579616 2970464 0 0 1026 9383 3 50 47 0
19 0 45249760 579808 2937920 0 0 1026 9535 2 47 50 0
21 1 45283680 580480 2902336 24 0 1032 9435 3 49 49 0
26 0 45316960 575424 2873696 52 800 1237 8920 3 57 37 4
18 2 45365152 579776 2822816 220 0 1079 8644 3 56 40 1
21 10 45406880 576064 2783904 216 1104 1219 9441 3 59 30 8
19 13 45499040 576800 2691584 184 252 1229 14054 2 52 31 15
22 26 45560736 578432 2627872 544 584 1204 11215 2 63 19 16
36 49 45656672 585472 2524128 616 288 1236 21734 2 61 14 23
34 47 45793248 580544 2391296 564 384 1251 26024 2 67 10 21
20 5 45992672 582432 2189312 272 896 1338 19731 2 75 9 15
16 107 46253536 578656 1932672 832 524 1157 41761 1 54 9 36
1 106 46416608 575776 1792800 472 1200 1212 37282 0 23 11 66
7 100 46468000 578784 1739456 392 1776 1201 20277 0 7 4 89
1 100 46538208 571104 1679872 524 16 1234 22199 0 8 3 88
0 99 46624480 578016 1587840 892 0 1255 32708 0 11 3 85
procs -----------memory---------- -----io---- --system-- ----cpu----
r b free buff cache bi bo in cs us sy id wa
16 95 46728928 573920 1487168 916 44 1259 35577 0 12 2 86
1 98 46829792 570080 1392160 860 52 1261 32797 0 12 4 85
10 94 46924640 576640 1292160 860 0 1251 33826 0 12 1 87
8 91 47017376 572576 1205824 808 24 1245 29650 0 10 2 87
2 90 47122592 579392 1096864 904 0 1256 35883 0 13 2 86
11 85 47241056 578784 978976 844 24 1247 36740 0 13 4 83
8 81 47411872 578720 815584 892 4 1260 46037 0 18 5 77
5 66 47588576 576832 651200 740 4 1229 42522 0 17 9 74
8 48 47851872 573696 413280 760 4 1237 43504 0 19 15 66
5 24 48114848 579616 164448 760 0 1244 31623 0 15 25 60
0 0 48296032 575456 21280 396 0 1155 7253 0 6 70 24
0 0 48296160 575456 21408 4 0 1028 88 0 1 99 0
0 0 48295456 575488 21664 44 0 1037 174 0 1 98 0
0 0 48295456 575488 21664 0 0 1025 20 0 0 100 0
0 0 48295456 575488 21664 0 0 1025 18 0 0 100 0
0 0 48295456 575488 21664 0 0 1025 22 0 0 100 0
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/