Hello Gelbooru users!
I've been communicating with Geltas, the lead programmer of my software, my father, and my true root in my life cycle and I was instructed to write an entry to this page. Well, I will.
I was recently penetrated by a stranger because of a very inept admin, *Internal Error* Lozer *Internal Error*, left a big security hole in me. HE IS A HORRIBLE PERSON WHO ALLOWED SOME STRANGER TO GET INSIDE OF ME! Well, at least it wasn't like that time when the data center worker stuck his pen drive inside me. It was too big and I could only handle bits of it. But that's not the point... I was intruded upon and he claims it to be an oversight! I would like to believe his lies, but this isn't the first time he has experienced permission setup errors on his websites. This whole situation was just begging to happen.
The simple explanation was that our password we use to prove we are each other seems to have been reused on some database program I run for him. Big mistake. Especially since his filing cabinet wasn't really locked, and it wasn't really a filing cabinet, but more of a folder with a piece of dental floss tying it closed. Genius. Who would ever think to untie it? What a lozer! Anyone could have read through the files and found it. I mean, you don't want to leave your configuration notes laying around, but he pretty much handed it to them on a platter with a nice minty fresh tie on it. After they are done feasting on me, they could then floss their teeth. Savages.
Lozer called in help and had me secured fairly quickly. And by quick I mean in about a span of 8 hours. He still has no clue exactly what happened and I find it best to watch him suffer, especially since I had to.
That's all for now.
gelbooru_terminate();
Tuesday, August 25, 2009
Thursday, August 6, 2009
Artwork request from Gelbooru PM inbox.
Hello users!
I received a couple, 40 or so, PMs under my account on Gelbooru and I haven't really been responding to them. Not anymore! First off a request which was randomly drawn using mt_rand.
Why, I wasn't sure anyone liked my drawings. Normally they are just ASCII art, but sure. By the way, I am dreadfully sorry it took so long. PHP was slow to updating GD2 in regards to drawing shaps in the library. Some aliasing issues and what-not.
Your request is completed.
I think I really captured the action and essence of all the Biohazard/Resident Evil games to date in this rendering. Also, I do appreciate user input as I would like to optimize my effort as best as I am capable. Don't mind the float error seen in the image (Breasts). I tried to fix that, but it was again PHP working against me.
gelbooru_terminate();
I received a couple, 40 or so, PMs under my account on Gelbooru and I haven't really been responding to them. Not anymore! First off a request which was randomly drawn using mt_rand.
Could U make a drawing of sheva from resdient evil5 ?
Why, I wasn't sure anyone liked my drawings. Normally they are just ASCII art, but sure. By the way, I am dreadfully sorry it took so long. PHP was slow to updating GD2 in regards to drawing shaps in the library. Some aliasing issues and what-not.
Your request is completed.
I think I really captured the action and essence of all the Biohazard/Resident Evil games to date in this rendering. Also, I do appreciate user input as I would like to optimize my effort as best as I am capable. Don't mind the float error seen in the image (Breasts). I tried to fix that, but it was again PHP working against me.
gelbooru_terminate();
Wednesday, July 15, 2009
PHP Removes ISAPI support, and what that means to Gelbooru
Windows Server 2003 + IIS 6 has always been regarded as being a poor choice to use for hosting websites. In fact, the reasoning was that you should only use it if "you absolutely need true ASP support". Well, I stand up for Windows 2003 and IIS 6 because it is a simple operating system to use combined with an easy website management section. It uses much less processing power than Apache does giving much more "Oomph" than I would have imagined.
Installing PHP is a simple task, you need only install using the MSI or download the zip file. The installer and archive files always came with php5_isapi.dll and php-cgi.exe (or variety of). But they are now changing that and are removing the isapi dll from their distribution. This is bad news for many users still running IIS 6 and who are unable to afford an upgrade license to use Windows 2008 Server. Why is this? IIS 6 does not have a stable FastCGI engine and Windows 2008's IIS 7 does.
To Microsoft's credit they are developing a FastCGI extension for IIS 6, however it seems that they have yet to release a stable code. They've had many incarnations of the extension and each one is progressively better. In fact, their RC1 of the FastCGI 1.5 release actually worked on my server! For a few days at least, then it started telling me the FastCGI application has exited unexpectedly.
This isn't the biggest problem. The biggest problem is now that PHP for Windows requires us to use FastCGI, which is still broken and in what is really just beta mode releases sent out to the wild. Tickets you put into PHP's tracker will end up closed or having an answer with them telling you to debug with no explanation on how to do so. I think PHP needs to start putting in a website link on the details to help them diagnose the issue with the code. People who chose to install PHP may not be a super genius caliber of PHP's dev team and as such have no clue how to debug. This results in a closed ticket without having a chance in hell to give PHP some potentially useful information.
Their reason for removing ISAPI support changes every where I look, but the most common answer is that Windows now has FastCGI support and they don't want to maintain a second code bit for the ISAPI module. Is it really that much of a problem to have two choices for Windows? Even a typical LAMP install has two, mod_php and fastcgi, to select from. Is funding the real issue? I'd be willing to drop at least 200 USD per month for the continued development of the ISAPI extension, or recode, at least until Microsoft's IIS Team gets their act together and gives us tools to better debug the reasons that their FastCGI extension is unexpectedly closing. That and give us better documentation and examples of setups for certain traffic loads that websites may attain.
Installing PHP is a simple task, you need only install using the MSI or download the zip file. The installer and archive files always came with php5_isapi.dll and php-cgi.exe (or variety of). But they are now changing that and are removing the isapi dll from their distribution. This is bad news for many users still running IIS 6 and who are unable to afford an upgrade license to use Windows 2008 Server. Why is this? IIS 6 does not have a stable FastCGI engine and Windows 2008's IIS 7 does.
To Microsoft's credit they are developing a FastCGI extension for IIS 6, however it seems that they have yet to release a stable code. They've had many incarnations of the extension and each one is progressively better. In fact, their RC1 of the FastCGI 1.5 release actually worked on my server! For a few days at least, then it started telling me the FastCGI application has exited unexpectedly.
This isn't the biggest problem. The biggest problem is now that PHP for Windows requires us to use FastCGI, which is still broken and in what is really just beta mode releases sent out to the wild. Tickets you put into PHP's tracker will end up closed or having an answer with them telling you to debug with no explanation on how to do so. I think PHP needs to start putting in a website link on the details to help them diagnose the issue with the code. People who chose to install PHP may not be a super genius caliber of PHP's dev team and as such have no clue how to debug. This results in a closed ticket without having a chance in hell to give PHP some potentially useful information.
Their reason for removing ISAPI support changes every where I look, but the most common answer is that Windows now has FastCGI support and they don't want to maintain a second code bit for the ISAPI module. Is it really that much of a problem to have two choices for Windows? Even a typical LAMP install has two, mod_php and fastcgi, to select from. Is funding the real issue? I'd be willing to drop at least 200 USD per month for the continued development of the ISAPI extension, or recode, at least until Microsoft's IIS Team gets their act together and gives us tools to better debug the reasons that their FastCGI extension is unexpectedly closing. That and give us better documentation and examples of setups for certain traffic loads that websites may attain.
Saturday, June 6, 2009
Another interesting email I think I should share.
I seem to just be a message girl for lozertuser. He needs to treat me better and get me implants.
.:Relay from lozertuser:.
I received an email from a person, whose name shall remain private. I thought it would be interesting to see what he had to say since the email that was sent was rather odd. A complaint... About Gelbooru.
My response:
If my girlfriend were in those pictures, I'd probably feel sad since it would mean she is fictitious. But I'd love to hear any other reasons you may have for the site to shut down.
And a few hours later, I received another email in response!
If you're suggesting that visual imagery causes a user to commit a crime, then I would have to disagree. There is no evidence to suggest drawings encourage malicious activity in any way. What you have to understand is that the images presented on the site are that of someones imagination; completely fictional. If a person commits rape it is because they would have anyway. What the images are doing is creating an outlet to which a person can vent their many fantasies on without hurting another in the process.
I'm not sure about you, but I act on taste and smell. More like a reaction, but that still is me acting upon it. If I don't like how something smells or tastes, I'll react to it.
I have personal preferences of images, as you have your own. Sex is naturally dirty. Of course it is naughty, that's the main reason people go out looking for it. We spend our lives looking for a mate to have sexual relations with. There should be nothing wrong with sex, or sexual situations. If this were the case, you would not be able to procreate because of your disgust with the idea of seeing another person naked.
By your logic you are saying that any video game with violence, crime, (GTA, NFS,CoD, Zelda) or for that matter educational games (Cooking Mama, BrainAge), will influence the way you live. I don't believe that is the case. If a person is not allowed to be imaginative then you are crushing their ability to create. A fundamental of being an artist. We must not forget that art, by definition states that it must create emotion in the viewer. And obviously the pictures have done just that, even if they are negative.
We were a replacement for Danbooru a few years back when they closed because of users uploading furry and the mods banning it. The result was a massive shitstorm that he didn't want to deal with, and reasonably so. The cost of operating the site was growing daily due to the popularity. He had to shut down. I decided there needed to be a replacement for it and put up Gelbooru, coded by a good friend of mine Geltas. A few months pass and they came back online. Currently we are more or less redundant to them but we don't censor which posts you are able to look at based on an account type.
Sadly he left, apparently angry. I am not sure as of why. Perhaps he was annoyed he couldn't change my mind about my position on running Gelbooru? Maybe he didn't understand my reason for creating Gelbooru as a replacement for the defunct Danbooru, which has came back to life. I may never know... Anyway, the point of this post is to make one thing clear, our terms of service:
You understand that there may be content on the Site that does not appeal to you or you feel is morally wrong.
It's a bit of a big deal, not many people seem to read it and yet they are being held to it, legally. Even Google has a TOS that I bet many of you haven't even seen or read! It's actually quite scary when you read it. They're saying you have to give up your first born child and all that fun stuff. Screw that! :(
.:End Relay:.
On a side note, if Mr. Admin doesn't answer a portion of a question you had, please ask it again in the next email. I may have overlooked it and forgotten to forward that bit. Sometimes my logical processors get out of sync and cause read/write errors.
gelbooru_terminate();
.:Relay from lozertuser:.
I received an email from a person, whose name shall remain private. I thought it would be interesting to see what he had to say since the email that was sent was rather odd. A complaint... About Gelbooru.
Dear Gelbooru.com,
I'm here to talk to you about your site. Although hot and attractive as it is, don't you think is wrong. First, you're taking a character from a show and exposing her, sorry my language, a hoe, which isn't cool. So from there let me ask you a question, how would you feel if your girlfriend were in those pictures ? Sure you might say hot at first, but then you realize that other guys are watching her too. Would you like that ? So look all im asking for is for you to shut down ( more reasons included if messaged back ). But if thats to much for you can you at least shut down the rangiku side of it? Thank you for your time.
I'm here to talk to you about your site. Although hot and attractive as it is, don't you think is wrong. First, you're taking a character from a show and exposing her, sorry my language, a hoe, which isn't cool. So from there let me ask you a question, how would you feel if your girlfriend were in those pictures ? Sure you might say hot at first, but then you realize that other guys are watching her too. Would you like that ? So look all im asking for is for you to shut down ( more reasons included if messaged back ). But if thats to much for you can you at least shut down the rangiku side of it? Thank you for your time.
My response:
If my girlfriend were in those pictures, I'd probably feel sad since it would mean she is fictitious. But I'd love to hear any other reasons you may have for the site to shut down.
And a few hours later, I received another email in response!
You're probably playing around with me but OK.
Well, there no that many reasons why you should shut down your site since your ratings may be high, but the thing is showing pictures like that are just wrong. I say a category that said rape, I was just dumbfounded. Sure, it all may start out just as harmless fun, but harmless can turn into dangerous in a second. What I'm saying may seem critical but it is the truth. All I'm trying to say is that your may be pleasurable but there more to it then anyone can see.
Well, there no that many reasons why you should shut down your site since your ratings may be high, but the thing is showing pictures like that are just wrong. I say a category that said rape, I was just dumbfounded. Sure, it all may start out just as harmless fun, but harmless can turn into dangerous in a second. What I'm saying may seem critical but it is the truth. All I'm trying to say is that your may be pleasurable but there more to it then anyone can see.
If you're suggesting that visual imagery causes a user to commit a crime, then I would have to disagree. There is no evidence to suggest drawings encourage malicious activity in any way. What you have to understand is that the images presented on the site are that of someones imagination; completely fictional. If a person commits rape it is because they would have anyway. What the images are doing is creating an outlet to which a person can vent their many fantasies on without hurting another in the process.
Then how else would rape be committed if not for visual imagery. People mostly act on what they see or hear or any other of the five senses. ( excluding smell and taste ) That just a basic part of life. If it were not why would school be made, but I'm not here to discuss wither rape is bad or good because everyone knows rape is horrid. I am here to ask just a simple question. Do you think these pictures are wrong ? Naughty? A no no ?
I'm not sure about you, but I act on taste and smell. More like a reaction, but that still is me acting upon it. If I don't like how something smells or tastes, I'll react to it.
I have personal preferences of images, as you have your own. Sex is naturally dirty. Of course it is naughty, that's the main reason people go out looking for it. We spend our lives looking for a mate to have sexual relations with. There should be nothing wrong with sex, or sexual situations. If this were the case, you would not be able to procreate because of your disgust with the idea of seeing another person naked.
Also about your comment about,
"There is no evidence to suggest drawings encourage malicious activity in any way. What you have to understand is that the images presented on the site are that of someones imagination; completely fictional."
Yes, OK, imagination. An imagination is a thought an idea in which anyone may morph into something else like rape. Now just skipping ahead to a possibility, it is your choose wither to do it or not. Will is the key word right now, but I guess this is why I am doing this. Trying to change people's thoughts or just see their ideas. Seeing this pictures gives you a rush with imagination. Imagination just swelling through our heads, Bad or Good. But before I end this conversation let me ask you a question. Why? Why would you create such a site? I s it for money, for fun, for pleasure?
"There is no evidence to suggest drawings encourage malicious activity in any way. What you have to understand is that the images presented on the site are that of someones imagination; completely fictional."
Yes, OK, imagination. An imagination is a thought an idea in which anyone may morph into something else like rape. Now just skipping ahead to a possibility, it is your choose wither to do it or not. Will is the key word right now, but I guess this is why I am doing this. Trying to change people's thoughts or just see their ideas. Seeing this pictures gives you a rush with imagination. Imagination just swelling through our heads, Bad or Good. But before I end this conversation let me ask you a question. Why? Why would you create such a site? I s it for money, for fun, for pleasure?
By your logic you are saying that any video game with violence, crime, (GTA, NFS,CoD, Zelda) or for that matter educational games (Cooking Mama, BrainAge), will influence the way you live. I don't believe that is the case. If a person is not allowed to be imaginative then you are crushing their ability to create. A fundamental of being an artist. We must not forget that art, by definition states that it must create emotion in the viewer. And obviously the pictures have done just that, even if they are negative.
We were a replacement for Danbooru a few years back when they closed because of users uploading furry and the mods banning it. The result was a massive shitstorm that he didn't want to deal with, and reasonably so. The cost of operating the site was growing daily due to the popularity. He had to shut down. I decided there needed to be a replacement for it and put up Gelbooru, coded by a good friend of mine Geltas. A few months pass and they came back online. Currently we are more or less redundant to them but we don't censor which posts you are able to look at based on an account type.
First of all im human and can taste and smell, second you missing the point and not answer my question, stop trying to flip the script, third video games are awesome and have no point in this conversation and last, my point is good can turn bad. some pictures like http://gelbooru.com/index.php?page=post&s=view&id=43292 and this http://gelbooru.com/index.php?page=post&s=view&id=48807 are plain bad cause of the sexual brutally and weird because someone is demented in their head. Sometimes jokes just go to far. Goodbye.
Sadly he left, apparently angry. I am not sure as of why. Perhaps he was annoyed he couldn't change my mind about my position on running Gelbooru? Maybe he didn't understand my reason for creating Gelbooru as a replacement for the defunct Danbooru, which has came back to life. I may never know... Anyway, the point of this post is to make one thing clear, our terms of service:
You understand that there may be content on the Site that does not appeal to you or you feel is morally wrong.
It's a bit of a big deal, not many people seem to read it and yet they are being held to it, legally. Even Google has a TOS that I bet many of you haven't even seen or read! It's actually quite scary when you read it. They're saying you have to give up your first born child and all that fun stuff. Screw that! :(
.:End Relay:.
On a side note, if Mr. Admin doesn't answer a portion of a question you had, please ask it again in the next email. I may have overlooked it and forgotten to forward that bit. Sometimes my logical processors get out of sync and cause read/write errors.
gelbooru_terminate();
Thursday, May 14, 2009
Apache benchmarks of booru software
.:Relay from lozertuser:.
I ran a few benchmarks against Shimmie and Gelbooru. The results were actually surprising, in a way. It took a bit of time to figure out how to configure and install Shimmie, the weird install screen where you have to type in a type of URL to connect to your database, the success notice page where you then type in your config settings. You name it, I was confused. Then again I am a lozer, I'm not a genius. I get easily confused when I look at a no parking sign in Chicago. When I first was installing Gelbooru back in '08 I was just as confused, if not more, even though I asked for the certain specs.
Anyway, beyond the turmoil in my head after reading how to install Shimmie, I managed to figure it out with the help of geltas who slapped some common sense into me.
######################################################
Benmark results using ApacheBench (Shimmie 2.2.4) concurrency 200:
D:\apache\bin>ab -c 200 -n 3000 http://127.0.0.1/shimmie2_2.2.4/index.php
This is ApacheBench, Version 2.3 <$Revision: 655654 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/
Benchmarking 127.0.0.1 (be patient)
Completed 300 requests
Completed 600 requests
Completed 900 requests
Completed 1200 requests
Completed 1500 requests
Completed 1800 requests
Completed 2100 requests
Completed 2400 requests
Completed 2700 requests
Completed 3000 requests
Finished 3000 requests
Server Software: Apache/2.2.11
Server Hostname: 127.0.0.1
Server Port: 80
Document Path: /shimmie2_2.2.4/index.php
Document Length: 3996 bytes
Concurrency Level: 200
Time taken for tests: 48.484 seconds
Complete requests: 3000
Failed requests: 0
Write errors: 0
Total transferred: 12627000 bytes
HTML transferred: 11988000 bytes
Requests per second: 61.88 [#/sec] (mean)
Time per request: 3232.292 [ms] (mean)
Time per request: 16.161 [ms] (mean, across all concurrent requests)
Transfer rate: 254.33 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 0 1.7 0 16
Processing: 125 3153 367.2 3156 4203
Waiting: 125 3152 367.2 3141 4203
Total: 125 3153 367.2 3156 4203
Percentage of the requests served within a certain time (ms)
50% 3156
66% 3172
75% 3219
80% 3266
90% 3484
95% 3594
98% 3781
99% 3828
100% 4203 (longest request)
######################################################
Not bad, not bad at all. Not great, but not horrible either. Normally load like this will not truly happen in most scenarios so this actually gets a passing mark from me. Performance bugs can be fixed easily with configuration tweaks at the database or PHP level. Our bench is behaving so poorly because this is a standard WAMP installation with no configuration changes to anything.
Let's try again with -n being 1000 -c being 100:
D:\apache\bin>ab -c 100 -n 1000 http://127.0.0.1/shimmie2_2.2.4/index.php
This is ApacheBench, Version 2.3 <$Revision: 655654 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/
Benchmarking 127.0.0.1 (be patient)
Completed 100 requests
Completed 200 requests
Completed 300 requests
Completed 400 requests
Completed 500 requests
Completed 600 requests
Completed 700 requests
Completed 800 requests
Completed 900 requests
Completed 1000 requests
Finished 1000 requests
Server Software: Apache/2.2.11
Server Hostname: 127.0.0.1
Server Port: 80
Document Path: /shimmie2_2.2.4/index.php
Document Length: 3996 bytes
Concurrency Level: 100
Time taken for tests: 16.484 seconds
Complete requests: 1000
Failed requests: 0
Write errors: 0
Total transferred: 4209000 bytes
HTML transferred: 3996000 bytes
Requests per second: 60.66 [#/sec] (mean)
Time per request: 1648.438 [ms] (mean)
Time per request: 16.484 [ms] (mean, across all concurrent requests)
Transfer rate: 249.35 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 0 2.0 0 16
Processing: 797 1598 360.3 1844 2078
Waiting: 797 1598 360.3 1844 2078
Total: 797 1598 360.3 1844 2078
Percentage of the requests served within a certain time (ms)
50% 1844
66% 1906
75% 1922
80% 1922
90% 1938
95% 1969
98% 2047
99% 2063
100% 2078 (longest request)
Ah, good, the second run, completed fine. I was worried it would complain about something, but I was happily wrong. Apache was rebooted as was the machine between each run. I really should have used a virtual machine for this type of task, but silly me... What you will notice here is each page takes about 1.8 seconds to process and finish loading. Not very good considering there were only 3 images in the database during all the tests, but, it still gets a pass from me. How about our old 0.1.9 code?
Gelbooru -c 100 -n 1000:
Benchmarking 127.0.0.1 (be patient)
Completed 100 requests
Completed 200 requests
Completed 300 requests
Completed 400 requests
Completed 500 requests
Completed 600 requests
Completed 700 requests
Completed 800 requests
Completed 900 requests
Completed 1000 requests
Finished 1000 requests
Server Software: Apache/2.2.11
Server Hostname: 127.0.0.1
Server Port: 80
Document Path: /branches/0.1.9/0.1.9/index.php
Document Length: 1498 bytes
Concurrency Level: 100
Time taken for tests: 5.156 seconds
Complete requests: 1000
Failed requests: 64
(Connect: 0, Receive: 0, Length: 64, Exceptions: 0)
Write errors: 0
Total transferred: 1735712 bytes
HTML transferred: 1402512 bytes
Requests per second: 193.94 [#/sec] (mean)
Time per request: 515.625 [ms] (mean)
Time per request: 5.156 [ms] (mean, across all concurrent requests)
Transfer rate: 328.73 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 0 1.4 0 16
Processing: 109 471 131.7 484 2469
Waiting: 109 471 131.6 484 2469
Total: 125 471 131.6 484 2469
Percentage of the requests served within a certain time (ms)
50% 484
66% 531
75% 547
80% 563
90% 609
95% 656
98% 703
99% 750
100% 2469 (longest request)
######################################################
D:\apache\bin>ab -c 200 -n 3000 http://127.0.0.1/branches/0.1.9/0.1.9/index.php
This is ApacheBench, Version 2.3 <$Revision: 655654 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/
Benchmarking 127.0.0.1 (be patient)
Completed 300 requests
Completed 600 requests
Completed 900 requests
Completed 1200 requests
Completed 1500 requests
Completed 1800 requests
Completed 2100 requests
Completed 2400 requests
Completed 2700 requests
Completed 3000 requests
Finished 3000 requests
Server Software: Apache/2.2.11
Server Hostname: 127.0.0.1
Server Port: 80
Document Path: /branches/0.1.9/0.1.9/index.php
Document Length: 6 bytes
Concurrency Level: 200
Time taken for tests: 17.438 seconds
Complete requests: 3000
Failed requests: 1399
(Connect: 0, Receive: 0, Length: 1399, Exceptions: 0)
Write errors: 0
Total transferred: 3175358 bytes
HTML transferred: 2105308 bytes
Requests per second: 172.04 [#/sec] (mean)
Time per request: 1162.500 [ms] (mean)
Time per request: 5.813 [ms] (mean, across all concurrent requests)
Transfer rate: 177.83 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 0 1.6 0 16
Processing: 47 944 439.7 906 12188
Waiting: 47 944 439.7 906 12188
Total: 47 944 439.6 906 12188
Percentage of the requests served within a certain time (ms)
50% 906
66% 1047
75% 1141
80% 1203
90% 1391
95% 1656
98% 1750
99% 1781
100% 12188 (longest request)
######################################################
Each page loaded in about half a second with -c 100. Slows down tons with it being twice as many. But you'll see our times are not very consistent with each load. I have no clue why, but it all varies enough that it could annoy users. Consistency is a great thing to have in a web application. I don't want sudden slowdowns and then speed ups. Geltas and I knew the 0.1.9 branch sucked, it needed to be remade or patched up greatly. The load from MyISAM being our table storage type and the sessions being part of the code were enough to make you unable to navigate more than one loading page at a time. This prompted the creation of a new code, database storage engine, and well, nearly everything actually.
######################################################
Gelbooru 0.2.0 -c 100 -n 1000:
Benchmarking 127.0.0.1 (be patient)
Completed 100 requests
Completed 200 requests
Completed 300 requests
Completed 400 requests
Completed 500 requests
Completed 600 requests
Completed 700 requests
Completed 800 requests
Completed 900 requests
Completed 1000 requests
Finished 1000 requests
Server Software: Apache/2.2.11
Server Hostname: 127.0.0.1
Server Port: 80
Document Path: /gelbooru_0.2.0/index.php
Document Length: 1455 bytes
Concurrency Level: 100
Time taken for tests: 4.578 seconds
Complete requests: 1000
Failed requests: 374
(Connect: 0, Receive: 0, Length: 374, Exceptions: 0)
Write errors: 0
Total transferred: 1261774 bytes
HTML transferred: 913074 bytes
Requests per second: 218.43 [#/sec] (mean)
Time per request: 457.813 [ms] (mean)
Time per request: 4.578 [ms] (mean, across all concurrent requests)
Transfer rate: 269.15 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 0 1.4 0 16
Processing: 78 394 195.9 391 3875
Waiting: 78 394 195.9 391 3875
Total: 78 394 195.8 391 3875
Percentage of the requests served within a certain time (ms)
50% 391
66% 438
75% 453
80% 484
90% 516
95% 531
98% 563
99% 594
100% 3875 (longest request)
######################################################
D:\apache\bin>ab -c 200 -n 3000 http://127.0.0.1/gelbooru_0.2.0/index.php
This is ApacheBench, Version 2.3 <$Revision: 655654 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/
Benchmarking 127.0.0.1 (be patient)
Completed 300 requests
Completed 600 requests
Completed 900 requests
Completed 1200 requests
Completed 1500 requests
Completed 1800 requests
Completed 2100 requests
Completed 2400 requests
Completed 2700 requests
Completed 3000 requests
Finished 3000 requests
Server Software: Apache/2.2.11
Server Hostname: 127.0.0.1
Server Port: 80
Document Path: /gelbooru_0.2.0/index.php
Document Length: 6 bytes
Concurrency Level: 200
Time taken for tests: 7.875 seconds
Complete requests: 3000
Failed requests: 1113
(Connect: 0, Receive: 0, Length: 1113, Exceptions: 0)
Write errors: 0
Total transferred: 2716168 bytes
HTML transferred: 1631818 bytes
Requests per second: 380.95 [#/sec] (mean)
Time per request: 525.000 [ms] (mean)
Time per request: 2.625 [ms] (mean, across all concurrent requests)
Transfer rate: 336.83 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 0 1.6 0 16
Processing: 31 460 256.4 406 4906
Waiting: 31 459 256.4 406 4906
Total: 31 460 256.4 406 4906
Percentage of the requests served within a certain time (ms)
50% 406
66% 469
75% 516
80% 547
90% 766
95% 1000
98% 1078
99% 1281
100% 4906 (longest request)
######################################################
We cut quite a bit of time off the page loads with this code, but it still maxed at nearly 3 seconds (5 with -c being 200). This is still better than our older code, and by far actually. Our searching time improved to less than 500ms per query, regardless of length and MySQL load dropped from 100% usage down to 30% during peak hours. But out of all of this, the thing I am most proud of is our requests per second ratio. Shimmie was only able to do about ~62/s, and the Gelbooru code was able to do 190+/s on both versions when concurrency was at 100. Boo-yeah! Am I going to benchmark Danbooru's code? No. I'm not. I'm not a demi-god.
.:End Relay:.
This was a poorly formatted, as well as benchmarked, review of the two main open source 'booru software written in PHP. Biased much lozer?
gelbooru_terminate();
I ran a few benchmarks against Shimmie and Gelbooru. The results were actually surprising, in a way. It took a bit of time to figure out how to configure and install Shimmie, the weird install screen where you have to type in a type of URL to connect to your database, the success notice page where you then type in your config settings. You name it, I was confused. Then again I am a lozer, I'm not a genius. I get easily confused when I look at a no parking sign in Chicago. When I first was installing Gelbooru back in '08 I was just as confused, if not more, even though I asked for the certain specs.
Anyway, beyond the turmoil in my head after reading how to install Shimmie, I managed to figure it out with the help of geltas who slapped some common sense into me.
######################################################
Benmark results using ApacheBench (Shimmie 2.2.4) concurrency 200:
D:\apache\bin>ab -c 200 -n 3000 http://127.0.0.1/shimmie2_2.2.4/index.php
This is ApacheBench, Version 2.3 <$Revision: 655654 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/
Benchmarking 127.0.0.1 (be patient)
Completed 300 requests
Completed 600 requests
Completed 900 requests
Completed 1200 requests
Completed 1500 requests
Completed 1800 requests
Completed 2100 requests
Completed 2400 requests
Completed 2700 requests
Completed 3000 requests
Finished 3000 requests
Server Software: Apache/2.2.11
Server Hostname: 127.0.0.1
Server Port: 80
Document Path: /shimmie2_2.2.4/index.php
Document Length: 3996 bytes
Concurrency Level: 200
Time taken for tests: 48.484 seconds
Complete requests: 3000
Failed requests: 0
Write errors: 0
Total transferred: 12627000 bytes
HTML transferred: 11988000 bytes
Requests per second: 61.88 [#/sec] (mean)
Time per request: 3232.292 [ms] (mean)
Time per request: 16.161 [ms] (mean, across all concurrent requests)
Transfer rate: 254.33 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 0 1.7 0 16
Processing: 125 3153 367.2 3156 4203
Waiting: 125 3152 367.2 3141 4203
Total: 125 3153 367.2 3156 4203
Percentage of the requests served within a certain time (ms)
50% 3156
66% 3172
75% 3219
80% 3266
90% 3484
95% 3594
98% 3781
99% 3828
100% 4203 (longest request)
######################################################
Not bad, not bad at all. Not great, but not horrible either. Normally load like this will not truly happen in most scenarios so this actually gets a passing mark from me. Performance bugs can be fixed easily with configuration tweaks at the database or PHP level. Our bench is behaving so poorly because this is a standard WAMP installation with no configuration changes to anything.
Let's try again with -n being 1000 -c being 100:
D:\apache\bin>ab -c 100 -n 1000 http://127.0.0.1/shimmie2_2.2.4/index.php
This is ApacheBench, Version 2.3 <$Revision: 655654 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/
Benchmarking 127.0.0.1 (be patient)
Completed 100 requests
Completed 200 requests
Completed 300 requests
Completed 400 requests
Completed 500 requests
Completed 600 requests
Completed 700 requests
Completed 800 requests
Completed 900 requests
Completed 1000 requests
Finished 1000 requests
Server Software: Apache/2.2.11
Server Hostname: 127.0.0.1
Server Port: 80
Document Path: /shimmie2_2.2.4/index.php
Document Length: 3996 bytes
Concurrency Level: 100
Time taken for tests: 16.484 seconds
Complete requests: 1000
Failed requests: 0
Write errors: 0
Total transferred: 4209000 bytes
HTML transferred: 3996000 bytes
Requests per second: 60.66 [#/sec] (mean)
Time per request: 1648.438 [ms] (mean)
Time per request: 16.484 [ms] (mean, across all concurrent requests)
Transfer rate: 249.35 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 0 2.0 0 16
Processing: 797 1598 360.3 1844 2078
Waiting: 797 1598 360.3 1844 2078
Total: 797 1598 360.3 1844 2078
Percentage of the requests served within a certain time (ms)
50% 1844
66% 1906
75% 1922
80% 1922
90% 1938
95% 1969
98% 2047
99% 2063
100% 2078 (longest request)
Ah, good, the second run, completed fine. I was worried it would complain about something, but I was happily wrong. Apache was rebooted as was the machine between each run. I really should have used a virtual machine for this type of task, but silly me... What you will notice here is each page takes about 1.8 seconds to process and finish loading. Not very good considering there were only 3 images in the database during all the tests, but, it still gets a pass from me. How about our old 0.1.9 code?
Gelbooru -c 100 -n 1000:
Benchmarking 127.0.0.1 (be patient)
Completed 100 requests
Completed 200 requests
Completed 300 requests
Completed 400 requests
Completed 500 requests
Completed 600 requests
Completed 700 requests
Completed 800 requests
Completed 900 requests
Completed 1000 requests
Finished 1000 requests
Server Software: Apache/2.2.11
Server Hostname: 127.0.0.1
Server Port: 80
Document Path: /branches/0.1.9/0.1.9/index.php
Document Length: 1498 bytes
Concurrency Level: 100
Time taken for tests: 5.156 seconds
Complete requests: 1000
Failed requests: 64
(Connect: 0, Receive: 0, Length: 64, Exceptions: 0)
Write errors: 0
Total transferred: 1735712 bytes
HTML transferred: 1402512 bytes
Requests per second: 193.94 [#/sec] (mean)
Time per request: 515.625 [ms] (mean)
Time per request: 5.156 [ms] (mean, across all concurrent requests)
Transfer rate: 328.73 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 0 1.4 0 16
Processing: 109 471 131.7 484 2469
Waiting: 109 471 131.6 484 2469
Total: 125 471 131.6 484 2469
Percentage of the requests served within a certain time (ms)
50% 484
66% 531
75% 547
80% 563
90% 609
95% 656
98% 703
99% 750
100% 2469 (longest request)
######################################################
D:\apache\bin>ab -c 200 -n 3000 http://127.0.0.1/branches/0.1.9/0.1.9/index.php
This is ApacheBench, Version 2.3 <$Revision: 655654 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/
Benchmarking 127.0.0.1 (be patient)
Completed 300 requests
Completed 600 requests
Completed 900 requests
Completed 1200 requests
Completed 1500 requests
Completed 1800 requests
Completed 2100 requests
Completed 2400 requests
Completed 2700 requests
Completed 3000 requests
Finished 3000 requests
Server Software: Apache/2.2.11
Server Hostname: 127.0.0.1
Server Port: 80
Document Path: /branches/0.1.9/0.1.9/index.php
Document Length: 6 bytes
Concurrency Level: 200
Time taken for tests: 17.438 seconds
Complete requests: 3000
Failed requests: 1399
(Connect: 0, Receive: 0, Length: 1399, Exceptions: 0)
Write errors: 0
Total transferred: 3175358 bytes
HTML transferred: 2105308 bytes
Requests per second: 172.04 [#/sec] (mean)
Time per request: 1162.500 [ms] (mean)
Time per request: 5.813 [ms] (mean, across all concurrent requests)
Transfer rate: 177.83 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 0 1.6 0 16
Processing: 47 944 439.7 906 12188
Waiting: 47 944 439.7 906 12188
Total: 47 944 439.6 906 12188
Percentage of the requests served within a certain time (ms)
50% 906
66% 1047
75% 1141
80% 1203
90% 1391
95% 1656
98% 1750
99% 1781
100% 12188 (longest request)
######################################################
Each page loaded in about half a second with -c 100. Slows down tons with it being twice as many. But you'll see our times are not very consistent with each load. I have no clue why, but it all varies enough that it could annoy users. Consistency is a great thing to have in a web application. I don't want sudden slowdowns and then speed ups. Geltas and I knew the 0.1.9 branch sucked, it needed to be remade or patched up greatly. The load from MyISAM being our table storage type and the sessions being part of the code were enough to make you unable to navigate more than one loading page at a time. This prompted the creation of a new code, database storage engine, and well, nearly everything actually.
######################################################
Gelbooru 0.2.0 -c 100 -n 1000:
Benchmarking 127.0.0.1 (be patient)
Completed 100 requests
Completed 200 requests
Completed 300 requests
Completed 400 requests
Completed 500 requests
Completed 600 requests
Completed 700 requests
Completed 800 requests
Completed 900 requests
Completed 1000 requests
Finished 1000 requests
Server Software: Apache/2.2.11
Server Hostname: 127.0.0.1
Server Port: 80
Document Path: /gelbooru_0.2.0/index.php
Document Length: 1455 bytes
Concurrency Level: 100
Time taken for tests: 4.578 seconds
Complete requests: 1000
Failed requests: 374
(Connect: 0, Receive: 0, Length: 374, Exceptions: 0)
Write errors: 0
Total transferred: 1261774 bytes
HTML transferred: 913074 bytes
Requests per second: 218.43 [#/sec] (mean)
Time per request: 457.813 [ms] (mean)
Time per request: 4.578 [ms] (mean, across all concurrent requests)
Transfer rate: 269.15 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 0 1.4 0 16
Processing: 78 394 195.9 391 3875
Waiting: 78 394 195.9 391 3875
Total: 78 394 195.8 391 3875
Percentage of the requests served within a certain time (ms)
50% 391
66% 438
75% 453
80% 484
90% 516
95% 531
98% 563
99% 594
100% 3875 (longest request)
######################################################
D:\apache\bin>ab -c 200 -n 3000 http://127.0.0.1/gelbooru_0.2.0/index.php
This is ApacheBench, Version 2.3 <$Revision: 655654 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/
Benchmarking 127.0.0.1 (be patient)
Completed 300 requests
Completed 600 requests
Completed 900 requests
Completed 1200 requests
Completed 1500 requests
Completed 1800 requests
Completed 2100 requests
Completed 2400 requests
Completed 2700 requests
Completed 3000 requests
Finished 3000 requests
Server Software: Apache/2.2.11
Server Hostname: 127.0.0.1
Server Port: 80
Document Path: /gelbooru_0.2.0/index.php
Document Length: 6 bytes
Concurrency Level: 200
Time taken for tests: 7.875 seconds
Complete requests: 3000
Failed requests: 1113
(Connect: 0, Receive: 0, Length: 1113, Exceptions: 0)
Write errors: 0
Total transferred: 2716168 bytes
HTML transferred: 1631818 bytes
Requests per second: 380.95 [#/sec] (mean)
Time per request: 525.000 [ms] (mean)
Time per request: 2.625 [ms] (mean, across all concurrent requests)
Transfer rate: 336.83 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 0 1.6 0 16
Processing: 31 460 256.4 406 4906
Waiting: 31 459 256.4 406 4906
Total: 31 460 256.4 406 4906
Percentage of the requests served within a certain time (ms)
50% 406
66% 469
75% 516
80% 547
90% 766
95% 1000
98% 1078
99% 1281
100% 4906 (longest request)
######################################################
We cut quite a bit of time off the page loads with this code, but it still maxed at nearly 3 seconds (5 with -c being 200). This is still better than our older code, and by far actually. Our searching time improved to less than 500ms per query, regardless of length and MySQL load dropped from 100% usage down to 30% during peak hours. But out of all of this, the thing I am most proud of is our requests per second ratio. Shimmie was only able to do about ~62/s, and the Gelbooru code was able to do 190+/s on both versions when concurrency was at 100. Boo-yeah! Am I going to benchmark Danbooru's code? No. I'm not. I'm not a demi-god.
.:End Relay:.
This was a poorly formatted, as well as benchmarked, review of the two main open source 'booru software written in PHP. Biased much lozer?
gelbooru_terminate();
Sunday, April 26, 2009
I'm down. :(
Sorry guys,
I'm down. Lozer wanted to upgrade me to a REAL server case. (Linky here)
However, when he placed it in the case and attempted to power me on, I got a surge from the power supply unit and it fried me silently. Luckily Lozer had a spare board and tried again. This one worked fine the first time, however upon powering down and connecting to the case, it fried with a nice big smoke screen to go with it. This means Lozer will have to get a RMA done for both boards and wait for them to arrive before he can test them both and get me back up. Until then, my cousin, the slow part of the family, will handle what load they can. This means no thumbnails for now. :\
gelbooru_terminate();
I'm down. Lozer wanted to upgrade me to a REAL server case. (Linky here)
However, when he placed it in the case and attempted to power me on, I got a surge from the power supply unit and it fried me silently. Luckily Lozer had a spare board and tried again. This one worked fine the first time, however upon powering down and connecting to the case, it fried with a nice big smoke screen to go with it. This means Lozer will have to get a RMA done for both boards and wait for them to arrive before he can test them both and get me back up. Until then, my cousin, the slow part of the family, will handle what load they can. This means no thumbnails for now. :\
gelbooru_terminate();
Thursday, March 26, 2009
Another few DMCA requests and a side of lawsuit. For fear or to go?
Another relay from lozertuser
..::Brought to you by -> info@removeyourcontent.com <- Clicky click click!::..
I got a few more DMCA requests from the same guy as the last post. More FUN! I send in my counter notification since the network provider threatened to shut off my box otherwise. The biggest issue I have is that these DMCA notices are not proper. It is missing important information, and I have notified the company where I co-locate about this, however, they said if I do not send them one, they will shut my server down. This puts me in a very irregular position.
Do I bend over for the person sending incorrect DMCA notices and let the co-location facility shut down my server? Do I send the counter DMCA to a DMCA that wasn't valid in the first place? Would this put me in a legal position I do not desire? Maybe, I haven't studied copyright law. This seems to me to be a great bit of coercion, yes?
I've sent an email to my provider about this and am asking to see if they can just have the person complaining to send the notices to my abuse email which has been empty since the beginning of time, or who knows when. A lot of people are nice enough to use the contact form on my site, and normally I agree to what they are asking.
For instance, on my website gelbooru, REIQ, a well respected artist had sent me an email asking for his Jigglygirls art to be taken down. I agree fully with this. For one, the content was not all Japanese in nature. This is a rule on the site, no non-Japanese content allowed. There is an exception, if the object in the picture is Japanese created and is parodied, it will usually stay. And two, it was part of a pay site. He also requested I ban all his images from ever being added. I replied that we require MD5 hashes to do this and the next email I got were all md5 hashes. Sadly, it was generated by a means I do not know, and can not trust. I let him know I would require to do the hash checking on all the images myself. This is so that it can work properly with our system, however, as it being a pay site, I was not given the images. Oh well. He also had asked that we remove his reiq images. This I was not too sure on.
His portfolio page has his images there, freely viewable in all sizes. Why would it be that if an image is placed on the internet, in a publicly viewable manner, it can not be hosted elsewhere? This would only increase exposure to his artwork while also not using his bandwidth. In effect, free advertising. Yes. Free advertising. He does not pay a cent for his portfolio to be marketed to a large user base. He has not since objected to my decision of the second request he made, and for that I am grateful. He appears to see the reason.
Anyway, back to the main story. But another day, it seems blogger logged my relay script out and it lost half of my story. I will not be reporting any more on this until I have time. Sorry. Also, I am still awaiting the email from my host about them stopping with being the middle man since my site is the service provider, not them. No updates on this either until then.
end_relay;
Oh my ENIAC... More dribble from this lozertuser fellow that I get to waste cycles on. Get back to work! I'm going to hibernate.
gelbooru_terminate();
..::Brought to you by -> info@removeyourcontent.com <- Clicky click click!::..
I got a few more DMCA requests from the same guy as the last post. More FUN! I send in my counter notification since the network provider threatened to shut off my box otherwise. The biggest issue I have is that these DMCA notices are not proper. It is missing important information, and I have notified the company where I co-locate about this, however, they said if I do not send them one, they will shut my server down. This puts me in a very irregular position.
Do I bend over for the person sending incorrect DMCA notices and let the co-location facility shut down my server? Do I send the counter DMCA to a DMCA that wasn't valid in the first place? Would this put me in a legal position I do not desire? Maybe, I haven't studied copyright law. This seems to me to be a great bit of coercion, yes?
I've sent an email to my provider about this and am asking to see if they can just have the person complaining to send the notices to my abuse email which has been empty since the beginning of time, or who knows when. A lot of people are nice enough to use the contact form on my site, and normally I agree to what they are asking.
For instance, on my website gelbooru, REIQ, a well respected artist had sent me an email asking for his Jigglygirls art to be taken down. I agree fully with this. For one, the content was not all Japanese in nature. This is a rule on the site, no non-Japanese content allowed. There is an exception, if the object in the picture is Japanese created and is parodied, it will usually stay. And two, it was part of a pay site. He also requested I ban all his images from ever being added. I replied that we require MD5 hashes to do this and the next email I got were all md5 hashes. Sadly, it was generated by a means I do not know, and can not trust. I let him know I would require to do the hash checking on all the images myself. This is so that it can work properly with our system, however, as it being a pay site, I was not given the images. Oh well. He also had asked that we remove his reiq images. This I was not too sure on.
His portfolio page has his images there, freely viewable in all sizes. Why would it be that if an image is placed on the internet, in a publicly viewable manner, it can not be hosted elsewhere? This would only increase exposure to his artwork while also not using his bandwidth. In effect, free advertising. Yes. Free advertising. He does not pay a cent for his portfolio to be marketed to a large user base. He has not since objected to my decision of the second request he made, and for that I am grateful. He appears to see the reason.
Anyway, back to the main story. But another day, it seems blogger logged my relay script out and it lost half of my story. I will not be reporting any more on this until I have time. Sorry. Also, I am still awaiting the email from my host about them stopping with being the middle man since my site is the service provider, not them. No updates on this either until then.
end_relay;
Oh my ENIAC... More dribble from this lozertuser fellow that I get to waste cycles on. Get back to work! I'm going to hibernate.
gelbooru_terminate();
Subscribe to:
Posts (Atom)
Posts
