Benchmark Results

Description

CS3911 Su mmer 2005 Monday 20 June 2005Team I I – A P Pr actice Exa m Daniel L ee OsieckiTechnical Memorandum:PHP & JSP Comparative BenchmarkOverviewThis do cument outlin es t he pro cedure a nd c onclusions of a compa rative b enchmark o f th eJSP an d PHP envi ronments r unning f unctionally i dentical c ode imple menting th e mostbasic func tionality of t he AP P ractice Exam s ystem. Th e g oal of this b enchmark is todemonstrate th e sup erior sc alability of th e PHP e nvironment i n th e s pecific p roblemdomain of th e AP Pra ctice Exa m.EnvironmentAll b enchmarks we re ru n on th e de velopers' pr ivate s ystem, a P entium I II cl ass L inuxserver e nvironment. I n a ll t ests th e d atabase an d w ebservices pl atform we re ru nsimultaneously on th e sa me ma chine.The o riginal J SP ve rsion of t he s ystem w as i nstalled and r un unde r A pacheJakarta/Tomcat v ersion 5.5.7 using J ava 1.4.2-01. This s ystem wa s insta lled e xactly asprovided wi th th e followin g c hanges:• Code d ependent on th e Mi crosoft Wi ndows op erating s ystem wa s c hanged to op eratein a s tandard c ross-platform ma nner.• Character ca se ir regularities we re fixed.• Database lo gin in formation was ch anged to int eract with the loc al da tabase se tup.The PHP v ersion o f th e s ystem w as imple mented b y the developers fo r th e pu rpose o f th ebenchmark. W herever possibl e s imilar cl ass a nd fun ction l ayouts w ere us ed to c reate aPHP ...

Informations

• Publié par: Mifog
• Nombre de lectures: 33
• Langue: English

Extrait

CS3911 Summer 2005 Team II – AP Practice Exam

Monday 20 June 2005 Daniel Lee Osiecki

Technical Memorandum: PHP & JSP Comparative Benchmark Overview This document outlines the procedure and conclusions of a comparative benchmark of the JSP and PHP environments running functionally identical code implementing the most basic functionality of the AP Practice Exam system.The goal of this benchmark is to demonstrate the superior scalability of the PHP environment in the specific problem domain of the AP Practice Exam. Environment All benchmarks were run on the developers' private system, a Pentium III class Linux server environment.In all tests the database and webservices platform were run simultaneously on the same machine. The original JSP version of the system was installed and run under Apache Jakarta/Tomcat version 5.5.7 using Java 1.4.2-01.This system was installed exactly as provided with the following changes:

· Code dependent on the Microsoft Windows operating system was changed to operate in a standard cross-platform manner. · Character case irregularities were fixed. · Database login information was changed to interact with the local database setup. The PHP version of the system was implemented by the developers for the purpose of the benchmark. Whereverpossible similar class and function layouts were used to create a PHP implementationthat operates as closely as possible to the JSP version.All outputs were made consistent or even identical with the original JSP version where possible.The aim of this prototype was not to be as efficient as possible, but to closely replicate the inner functionality of the original version. Procedure Benchmarks were performed from the command line using the time command to measure total time of execution of thewgetcommand. The wget command is a very simple program that retrieves a URL from a web server.To ensure minimal influence of the testing tool on the benchmark results, output was sent to /dev/null . Individual test runs were performed with the following commands:

· PHP - time wget -O /dev/null "http://127.0.0.1/~cs3911/questionForBenchmark.phtml?maxQuest=1 &examType=1&mode=question&questNum=1&forceInitDB=true" · JSP - time wget -O /dev/null "http://127.0.0.1:8090/apExam/begin.jsp?numQuestions=1&examType =1"

The results here use the "real" component of time measurement, as it measures the complete time to execute the command, which closely measures the time until the requested page is fully loaded in the user's browser. Both implementations were measured for scalability by requesting the first question in the exam series and varying the number of total questions requested.This operation was chosen because it is the most data-intensive and historically problematic operation with respect to scalability.Tests were run 5 times each for the following maximum number of questions: 1, 5, 10, 15, 20, 25, 30, 35, 40, 50, 60. Results Full results may be found in appendix A.The summary results are presented here. Both implementations showed a trend of linear increase of response time when more questions were requested.The difference in scaling factor, however, was very large - the JSP version scaled upwards more than 25 times more quickly than the PHP version.

The following graph shows these trends:

Figure 1 - PHP vs. JSP Implementation Scaling Comparison

Appendix A: Detailed Results Total PHPResponse JSPResponse (average) (average) Questions Time(s) Time(s) 60 0.2044.637 60 0.2054.594 60 0.2040.205 4.5804.605 60 0.2074.609 60 0.2054.603 50 0.1773.507 50 0.1774.038 50 0.1760.174 4.0143.925 50 0.1614.029 50 0.1794.035 40 0.1463.316 40 0.1502.806 40 0.1480.147 3.3383.128 40 0.1463.364 40 0.1472.814 35 0.1333.001 35 0.1332.969 35 0.1340.134 2.4962.889 35 0.1352.983 35 0.1332.994 30 0.1192.132 30 0.1172.657 30 0.1180.118 2.1242.267 30 0.1172.145 30 0.1192.276 25 0.1042.273 25 0.1051.810 25 0.1050.103 2.3002.117 25 0.0971.868 25 0.1032.335 20 0.0911.459 20 0.0911.455 20 0.0890.090 1.9801.682 20 0.0891.487

20 15 15 15 15 15 10 10 10 10 10 5 5 5 5 5 1 1 1 1 1

0.091 0.075 0.075 0.076 0.077 0.075 0.062 0.062 0.062 0.062 0.062 0.048 0.047 0.047 0.051 0.048 0.036 0.036 0.036 0.036 0.036

0.076

0.062

0.048

0.036

2.029 1.319 1.156 1.136 1.650 1.208 0.944 0.803 0.791 0.782 0.784 0.402 0.407 0.416 0.401 0.407 0.096 0.113 0.120 0.111 0.095

1.294

0.821

0.407

0.107