.:Relay from lozertuser:.
I ran a few benchmarks against Shimmie and Gelbooru. The results were actually surprising, in a way. It took a bit of time to figure out how to configure and install Shimmie, the weird install screen where you have to type in a type of URL to connect to your database, the success notice page where you then type in your config settings. You name it, I was confused. Then again I am a lozer, I'm not a genius. I get easily confused when I look at a no parking sign in Chicago. When I first was installing Gelbooru back in '08 I was just as confused, if not more, even though I asked for the certain specs.
Anyway, beyond the turmoil in my head after reading how to install Shimmie, I managed to figure it out with the help of geltas who slapped some common sense into me.
######################################################
Benmark results using ApacheBench (Shimmie 2.2.4) concurrency 200:
D:\apache\bin>ab -c 200 -n 3000 http://127.0.0.1/shimmie2_2.2.4/index.php
This is ApacheBench, Version 2.3 <$Revision: 655654 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/
Benchmarking 127.0.0.1 (be patient)
Completed 300 requests
Completed 600 requests
Completed 900 requests
Completed 1200 requests
Completed 1500 requests
Completed 1800 requests
Completed 2100 requests
Completed 2400 requests
Completed 2700 requests
Completed 3000 requests
Finished 3000 requests
Server Software: Apache/2.2.11
Server Hostname: 127.0.0.1
Server Port: 80
Document Path: /shimmie2_2.2.4/index.php
Document Length: 3996 bytes
Concurrency Level: 200
Time taken for tests: 48.484 seconds
Complete requests: 3000
Failed requests: 0
Write errors: 0
Total transferred: 12627000 bytes
HTML transferred: 11988000 bytes
Requests per second: 61.88 [#/sec] (mean)
Time per request: 3232.292 [ms] (mean)
Time per request: 16.161 [ms] (mean, across all concurrent requests)
Transfer rate: 254.33 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 0 1.7 0 16
Processing: 125 3153 367.2 3156 4203
Waiting: 125 3152 367.2 3141 4203
Total: 125 3153 367.2 3156 4203
Percentage of the requests served within a certain time (ms)
50% 3156
66% 3172
75% 3219
80% 3266
90% 3484
95% 3594
98% 3781
99% 3828
100% 4203 (longest request)
######################################################
Not bad, not bad at all. Not great, but not horrible either. Normally load like this will not truly happen in most scenarios so this actually gets a passing mark from me. Performance bugs can be fixed easily with configuration tweaks at the database or PHP level. Our bench is behaving so poorly because this is a standard WAMP installation with no configuration changes to anything.
Let's try again with -n being 1000 -c being 100:
D:\apache\bin>ab -c 100 -n 1000 http://127.0.0.1/shimmie2_2.2.4/index.php
This is ApacheBench, Version 2.3 <$Revision: 655654 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/
Benchmarking 127.0.0.1 (be patient)
Completed 100 requests
Completed 200 requests
Completed 300 requests
Completed 400 requests
Completed 500 requests
Completed 600 requests
Completed 700 requests
Completed 800 requests
Completed 900 requests
Completed 1000 requests
Finished 1000 requests
Server Software: Apache/2.2.11
Server Hostname: 127.0.0.1
Server Port: 80
Document Path: /shimmie2_2.2.4/index.php
Document Length: 3996 bytes
Concurrency Level: 100
Time taken for tests: 16.484 seconds
Complete requests: 1000
Failed requests: 0
Write errors: 0
Total transferred: 4209000 bytes
HTML transferred: 3996000 bytes
Requests per second: 60.66 [#/sec] (mean)
Time per request: 1648.438 [ms] (mean)
Time per request: 16.484 [ms] (mean, across all concurrent requests)
Transfer rate: 249.35 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 0 2.0 0 16
Processing: 797 1598 360.3 1844 2078
Waiting: 797 1598 360.3 1844 2078
Total: 797 1598 360.3 1844 2078
Percentage of the requests served within a certain time (ms)
50% 1844
66% 1906
75% 1922
80% 1922
90% 1938
95% 1969
98% 2047
99% 2063
100% 2078 (longest request)
Ah, good, the second run, completed fine. I was worried it would complain about something, but I was happily wrong. Apache was rebooted as was the machine between each run. I really should have used a virtual machine for this type of task, but silly me... What you will notice here is each page takes about 1.8 seconds to process and finish loading. Not very good considering there were only 3 images in the database during all the tests, but, it still gets a pass from me. How about our old 0.1.9 code?
Gelbooru -c 100 -n 1000:
Benchmarking 127.0.0.1 (be patient)
Completed 100 requests
Completed 200 requests
Completed 300 requests
Completed 400 requests
Completed 500 requests
Completed 600 requests
Completed 700 requests
Completed 800 requests
Completed 900 requests
Completed 1000 requests
Finished 1000 requests
Server Software: Apache/2.2.11
Server Hostname: 127.0.0.1
Server Port: 80
Document Path: /branches/0.1.9/0.1.9/index.php
Document Length: 1498 bytes
Concurrency Level: 100
Time taken for tests: 5.156 seconds
Complete requests: 1000
Failed requests: 64
(Connect: 0, Receive: 0, Length: 64, Exceptions: 0)
Write errors: 0
Total transferred: 1735712 bytes
HTML transferred: 1402512 bytes
Requests per second: 193.94 [#/sec] (mean)
Time per request: 515.625 [ms] (mean)
Time per request: 5.156 [ms] (mean, across all concurrent requests)
Transfer rate: 328.73 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 0 1.4 0 16
Processing: 109 471 131.7 484 2469
Waiting: 109 471 131.6 484 2469
Total: 125 471 131.6 484 2469
Percentage of the requests served within a certain time (ms)
50% 484
66% 531
75% 547
80% 563
90% 609
95% 656
98% 703
99% 750
100% 2469 (longest request)
######################################################
D:\apache\bin>ab -c 200 -n 3000 http://127.0.0.1/branches/0.1.9/0.1.9/index.php
This is ApacheBench, Version 2.3 <$Revision: 655654 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/
Benchmarking 127.0.0.1 (be patient)
Completed 300 requests
Completed 600 requests
Completed 900 requests
Completed 1200 requests
Completed 1500 requests
Completed 1800 requests
Completed 2100 requests
Completed 2400 requests
Completed 2700 requests
Completed 3000 requests
Finished 3000 requests
Server Software: Apache/2.2.11
Server Hostname: 127.0.0.1
Server Port: 80
Document Path: /branches/0.1.9/0.1.9/index.php
Document Length: 6 bytes
Concurrency Level: 200
Time taken for tests: 17.438 seconds
Complete requests: 3000
Failed requests: 1399
(Connect: 0, Receive: 0, Length: 1399, Exceptions: 0)
Write errors: 0
Total transferred: 3175358 bytes
HTML transferred: 2105308 bytes
Requests per second: 172.04 [#/sec] (mean)
Time per request: 1162.500 [ms] (mean)
Time per request: 5.813 [ms] (mean, across all concurrent requests)
Transfer rate: 177.83 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 0 1.6 0 16
Processing: 47 944 439.7 906 12188
Waiting: 47 944 439.7 906 12188
Total: 47 944 439.6 906 12188
Percentage of the requests served within a certain time (ms)
50% 906
66% 1047
75% 1141
80% 1203
90% 1391
95% 1656
98% 1750
99% 1781
100% 12188 (longest request)
######################################################
Each page loaded in about half a second with -c 100. Slows down tons with it being twice as many. But you'll see our times are not very consistent with each load. I have no clue why, but it all varies enough that it could annoy users. Consistency is a great thing to have in a web application. I don't want sudden slowdowns and then speed ups. Geltas and I knew the 0.1.9 branch sucked, it needed to be remade or patched up greatly. The load from MyISAM being our table storage type and the sessions being part of the code were enough to make you unable to navigate more than one loading page at a time. This prompted the creation of a new code, database storage engine, and well, nearly everything actually.
######################################################
Gelbooru 0.2.0 -c 100 -n 1000:
Benchmarking 127.0.0.1 (be patient)
Completed 100 requests
Completed 200 requests
Completed 300 requests
Completed 400 requests
Completed 500 requests
Completed 600 requests
Completed 700 requests
Completed 800 requests
Completed 900 requests
Completed 1000 requests
Finished 1000 requests
Server Software: Apache/2.2.11
Server Hostname: 127.0.0.1
Server Port: 80
Document Path: /gelbooru_0.2.0/index.php
Document Length: 1455 bytes
Concurrency Level: 100
Time taken for tests: 4.578 seconds
Complete requests: 1000
Failed requests: 374
(Connect: 0, Receive: 0, Length: 374, Exceptions: 0)
Write errors: 0
Total transferred: 1261774 bytes
HTML transferred: 913074 bytes
Requests per second: 218.43 [#/sec] (mean)
Time per request: 457.813 [ms] (mean)
Time per request: 4.578 [ms] (mean, across all concurrent requests)
Transfer rate: 269.15 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 0 1.4 0 16
Processing: 78 394 195.9 391 3875
Waiting: 78 394 195.9 391 3875
Total: 78 394 195.8 391 3875
Percentage of the requests served within a certain time (ms)
50% 391
66% 438
75% 453
80% 484
90% 516
95% 531
98% 563
99% 594
100% 3875 (longest request)
######################################################
D:\apache\bin>ab -c 200 -n 3000 http://127.0.0.1/gelbooru_0.2.0/index.php
This is ApacheBench, Version 2.3 <$Revision: 655654 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/
Benchmarking 127.0.0.1 (be patient)
Completed 300 requests
Completed 600 requests
Completed 900 requests
Completed 1200 requests
Completed 1500 requests
Completed 1800 requests
Completed 2100 requests
Completed 2400 requests
Completed 2700 requests
Completed 3000 requests
Finished 3000 requests
Server Software: Apache/2.2.11
Server Hostname: 127.0.0.1
Server Port: 80
Document Path: /gelbooru_0.2.0/index.php
Document Length: 6 bytes
Concurrency Level: 200
Time taken for tests: 7.875 seconds
Complete requests: 3000
Failed requests: 1113
(Connect: 0, Receive: 0, Length: 1113, Exceptions: 0)
Write errors: 0
Total transferred: 2716168 bytes
HTML transferred: 1631818 bytes
Requests per second: 380.95 [#/sec] (mean)
Time per request: 525.000 [ms] (mean)
Time per request: 2.625 [ms] (mean, across all concurrent requests)
Transfer rate: 336.83 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 0 1.6 0 16
Processing: 31 460 256.4 406 4906
Waiting: 31 459 256.4 406 4906
Total: 31 460 256.4 406 4906
Percentage of the requests served within a certain time (ms)
50% 406
66% 469
75% 516
80% 547
90% 766
95% 1000
98% 1078
99% 1281
100% 4906 (longest request)
######################################################
We cut quite a bit of time off the page loads with this code, but it still maxed at nearly 3 seconds (5 with -c being 200). This is still better than our older code, and by far actually. Our searching time improved to less than 500ms per query, regardless of length and MySQL load dropped from 100% usage down to 30% during peak hours. But out of all of this, the thing I am most proud of is our requests per second ratio. Shimmie was only able to do about ~62/s, and the Gelbooru code was able to do 190+/s on both versions when concurrency was at 100. Boo-yeah! Am I going to benchmark Danbooru's code? No. I'm not. I'm not a demi-god.
.:End Relay:.
This was a poorly formatted, as well as benchmarked, review of the two main open source 'booru software written in PHP. Biased much lozer?
gelbooru_terminate();
Subscribe to:
Post Comments (Atom)
Posts
No comments:
Post a Comment