Validating Web Performance Levels with Benchmark Factory

Wayne_Chan

Sujets

• Quest Software
• Gestion de configuration logicielle
• Linternaute.com
• Agent orange
• Loadstar: The Legend of Tully Bodine
• Tune In, Tokyo...

Informations

• Publié par: Zowyug
• Nombre de lectures: 92
• Langue: Slovak

Extrait

Validating Web Performance Levels with

Benchmark Factory

®

By Bernard Farrell and Kevin Dalton

Contents

Introduction ...............................................................................................................................................................3

Key Web Performance Testing Terminology .............................................................................................................3

Benchmarking..............................................................................................................................................3

Capacity Planning.......................................................................................................................................3

Load Testing................................................................................................................................................4

Stress Testing...............................................................................................................................................4

Web Environments and Testing Complexities............................................................................................................4

Identifying Critical Risk Areas...................................................................................................................................4

Technologies Involved during Web Testing...............................................................................................................5

Benchmark Factory®.................................................................................................................................................5

Benchmark Factory Core Components......................................................................................................................6

Visual Control Center..................................................................................................................................6

Agent............................................................................................................................................................6

Repository....................................................................................................................................................6

Benchmark Factory Modules.....................................................................................................................................8

Database Module.........................................................................................................................................9

Internet Module ...........................................................................................................................................9

Messaging Module.......................................................................................................................................9

File Server Module......................................................................................................................................9

Web Analysis and Profiling with Benchmark Factory...............................................................................................9

Real-Time Diagnostics.............................................................................................................................................10

Web Tuning..............................................................................................................................................................10

Conclusion ...............................................................................................................................................................10

About the Authors ....................................................................................................................................................11

About Quest Software ..............................................................................................................................................11

2

Validating Web Performance Levels with Benchmark Factory

®

By Bernard Farrell and Kevin Dalton

Introduction

The level of business conducted on the World Wide Web (WWW) continues to grow at

a tremendous rate. Consequently, business-to-consumer (B2C) and business-to-business

(B2B) arenas, equate to increase sales and business opportunities. Change is constant

in these arenas. Those who quickly adapt to change, profit. Those who fail to change,

struggle to stay competitive and survive. Productivity, efficiency, and customer

satisfaction require efficient Web technology. Successful e-business environments

demand Web performance tracking, validation, and evaluation.

Excellent Web performance equates to excellent customer service. Customer service

equates to maximize profits. In the highly competitive and unforgiving Internet e-

commerce environment, server performance is the defining factor that keeps customers

satisfied. Satisfied customers, in turn, provide higher company revenue and profit.

Benchmark Factory

®

is a load testing and capacity planning tool uniquely designed for

critical e-business environments. Benchmark Factory places enormous stress on a

system-under-test that typically is hard to achieve in a standard testing environment.

Systems may fail under extreme loads. Benchmark Factory identifies system capacity

and performance bottlenecks before they occur reducing downtime, development costs,

and potential loss of revenue.

Key Web Performance Testing Terminology

Web performance testing terms include:

Benchmarking

•

•

•

•

Capacity planning

Load testing

Stress testing

Benchmarking

Benchmarking is the performance test of hardware and/or software. Benchmark

Factory software can accurately test the raw power of a single machine, the interaction

in a single client/server system (one server/multiple clients), and the transactions per

second in a transaction processing system.

Capacity Planning

Capacity planning measures computer utilization of a system’s software and

applications. Capacity planners measure this computer utilization based on simulation

or mathematical models. Simulation tools like Benchmark Factory create mock

networks to gauge usage. Mathematical tools use formulas developed from previous

3

performance data to calculate expected resource consumption. Capacity planners

measure the use of host CPU, memory, and disk storage for mainframe machines. In

addition, capacity planners can measure database and network activity for server and

client machines.

Load Testing

Load testing determines system performance at maximum usage levels. The measure

of transaction times during load-testing is not as important as determining how far a

system can be pushed until it fails.

Stress Testing

Stress testing runs or stresses a program under heavy loads for sustained periods. Stress

testing determines when the performance of a system degrades over a period of time.

Web Environments and Testing Complexities

Web sites vary in complexity. Simple sites may use only static pages with a few

graphics. Sites that are more complex utilize interactive features, dynamic

(customized) content, or database connectivity. Enterprise Web traffic involves

complex interchanges between Web browsers, programming languages, operating

systems, communications protocols, servers, databases, plug-ins, security

enhancements, and firewalls. These complexities present the challenge of focusing on

what testing to perform.

In addition to Web site complexity, consider the testing environment and how it

parallels true “production” environment testing. Testing scenarios require replicated

real-life users, Internet connections, modems, communications, hardware, databases, or

other factors peculiar to the testing site. Web performance testing should strive to

duplicate a “production” system. Production differences must be considered when

developing testing plans.

Identifying Critical Risk Areas

Reviewing critical risk areas on a Web site is vital when validating Web performance

levels. Carefully consider factors that affect customer satisfaction and income

potential. The following provides a list of critical areas to review during planning

stages:

What types of problems cause the most complaints or negative publicity

•

•

•

•

•

What are the interactions between HyperText Markup Language (HTML) pages

What aspects of a site's Common Gateway Interface (CGI), applets, ActiveX

components, database interfaces, Active Server Page (ASP), and dynamic page

generators, are most complex

What Web site functionality is most critical to its purpose

What site areas require the heaviest database interaction

4

What site areas are the most popular and generate the most traffic

•

•

•

•

•

•

•

•

•

•

•

•

•

•

•

•

•

•

•

•

•

•

What are the TCP/IP communications/firewalls requirements

There are a variety of servers and browsers used for Web e-commerce activity.

Numerous versions of servers and browsers, variations in connection speeds, rapidly

changing technologies, and multiple standards and protocols create testing challenges,

and force major, ongoing Web site testing.

Technologies Involved during Web Testing

Carefully consider the technologies used on a Web site. Reviewing and understanding

the interaction of applicable site technologies provides the basis for obtaining accurate

testing results. Benchmark Factory supports the following technologies:

Static HTML

Dynamic HTML

Multimedia

Common Gateway Interface (CGI)

Application Server Page (ASP)

JavaServer Page (JSP)

Hypertext Preprocessor (PHP)

JavaScript

Perl

VB Script

Java DataBase Connectivity (JDBC)

Java

Plug-ins

Structured Query Language (SQL)

Proxies

Secure Sockets Layer (SSL)

Web servers

Application servers

Cache servers

Load balancers

Benchmark Factory

Benchmark Factory provides all of the essential functions necessary to test an e-

commerce application by developing custom server benchmarks. Simulated throughput

provides an overall measure of server performance, including server software and

hardware performance; throughput performance; and total system throughput.

The operations contained within Benchmark Factory represent real applications doing

real work. They contain specially written transactions (tests) that mimic real-world

5

client/server application transactions, without actually running the client side

application.

Benchmark Factory provides an accurate method of collecting and measuring

transactions per second (TPS) and other system-under-test statistics. Benchmark

Factory allows a client/server developer to create custom benchmarks. Properly

developed custom benchmarks, allow testers to draw conclusions about server

performance, transaction design, and overall solution stability. Benchmark Factory

supports a variety of server systems, including client/server database servers, file

servers, messaging servers, transaction servers, cache servers, and Internet servers.

Benchmark Factory Core Components

Benchmark Factory consists of the following core components:

Visual Control Center

•

•

•

Agent

Repository

These core components are multi-threaded 32-bit Windows applications capable of

running on Windows NT and Windows 2000 platforms.

Visual Control Center

The Visual Control Center is the dashboard of the Benchmark Factory testing

environment. It is where test scenarios run and are coordinated. The Visual Control

Center monitors test progress, collects statistics, and performs data analysis.

Agent

Agents are workstations that run the Agent software. The Agent software sends

requests to a system-under-test. Agents are multi-threaded, and simulate more than a

thousand users on a single computer. Benchmark Factory is infinitely scaleable. It

scales horizontally (supports hundreds of workstations running the Agent software),

and vertically (each Agent is capable of running a thousand threads.)

Repository

The repository stores database results. The Visual Control Center inserts testing results

into the repository and provides access to its data. In addition, the repository houses all

test scenarios with settings and options. By default, the repository is a Sybase SQL

Anywhere database that resides on the same machine as the Visual Control Center. If

required, the repository can be changed to a Microsoft SQL Server database.

6

Figure 1 provides an illustration of Benchmark Factory components.

Figure 1—Benchmark Factory Core Components

Benchmark Factory collects a vast amount of statistics, such as overall server

throughput (measured in transactions per second/minute, or bytes transferred), and

detailed transactions executed statistics by individual workstation producing a testing

load. These statistics measure, analyze, and predict the capacity of a server system.

Benchmark Factory software is a stand-alone benchmarking program that loads on a

server or workstation and does not require third party or custom applications.

7

Benchmark Factory Modules

Benchmark Factory provides the versatility to benchmark all testing platforms

adequately. Companies depend on the Web for revenue. Web site downtime quickly

leads to a loss of revenue, poor user perception and loss of customer loyalty.

Companies can determine future hardware and software investments to meet expected

e-commerce demands by performing adequate capacity planning.

Quest Software meets the wide range of Web performance testing requirements by

packaging Benchmark Factory as a complete suite of testing platforms, or by allowing

the purchase of individual platforms (Figure 2.) Benchmark Factory supports the

following testing platforms:

Database

•

•

•

•

Internet

Messaging

File Server

Figure 2—Benchmark Factory Modules

Unlike other testing tools, Benchmark Factory creates realistic tests without

programming and provides comprehensive, easy-to-understand reports.

8

Database Module

The Benchmark Factory Database module allows IT managers and database

administrators to ensure their database applications achieve the required service levels

of performance and availability by determining true database performance, and

identifying the point-of-system failure.

Internet Module

The Benchmark Factory Internet module allows Web site administrators to ensure their

Web applications achieve required service levels of performance, availability, and

identify the point-of-system failure.

Messaging Module

The Benchmark Factory Messaging module allows messaging administrators to ensure their

email servers achieve the required service levels of performance, availability, and identifies the

point-of-system failure.

File Server Module

The Benchmark Factory Internet module allows IT Managers to ensure their file servers

achieve the required service levels of performance and identifies the point-of-system

failure.

Web Analysis and Profiling with Benchmark Factory

Benchmark Factory records essential performance metrics during each test.

Administrators use these metrics to determine the most effective course of action to

improve performance. Benchmark Factory illuminates the behavior of a system-under-

test to completely review a system’s architecture.

Benchmark Factory provides modeling and reporting abilities, making it an invaluable

tool for measuring server performance. Creating realistic users and executing

transactions against a server becomes a trivial task. The addition of script functions

expands a tester’s ability to simulate further realistic user models. Modeling realistic

users and measuring the transaction times and throughput, at different user loads,

presents a clear picture of system capacity. Benchmark Factory gathers numerous

metrics that include:

Response time

•

•

•

•

•

Retrieval time

Transaction time

Transactions per second

Bytes per second

9

Real-Time Diagnostics

Benchmark Factory produces real-time graphs and raw data when running a test.

During initial benchmarking, real-time analysis provides adequate information to

perform minor tuning and initial capacity estimates. These real-time graphs and raw

data may be sufficient to identify obvious transaction problems, or estimate maximum

user load.

Web Tuning

The process of using Benchmark Factory to tune a Web server involves the following

steps:

1. Running a test against a Web server

2. Reviewing the recorded metrics and other performance factors

3. Tuning a server to improve performance

This test cycle is summarized as; test, tune, and repeat. The following provides real-

world testing examples:

An administrator notices a particular Web server has performance problems and feels it

is memory related. An administrator runs a test against a server. After completing a

test, an administrator increases the physical memory in a machine. Next, an

administrator runs the same test again and compares the test results

•

•

•

•

•

An administrator has an interactive Web site with a database used to put dynamic

information into the Web pages. The database resides on the same system as the Web

server. An administrator feels that the database could scale to higher user loads if the

processes were on separate servers. To test this theory, an administrator tests a server

at different user loads, then splits the database off to another server and re-runs a test

using the same user loads. An administrator then compares the results to determine

where the two architectures diverge in performance

Conclusion

Use the best tools available when benchmarking e-commerce sites. An effective e-

commerce site must scale with the number of users, respond reliably and quickly, and

optimizes its use of bandwidth. Benchmark Factory provides all of the essential

functions necessary to test e-commerce applications. In addition to Benchmark

Factory, Quest Software offers the following tools to monitor and tune e-commerce

systems during capacity planning, load-testing, and normal operation:

Spotlight

®

—Provides essential functions to tune specific systems. Quest Software

offers Spotlight for Oracle, SQL servers, and Web servers

I/Watch™

—Provides essential functions to monitor and tune Oracle and SQL Server

databases with trend analysis and drill-down capabilities

Foglight®

—Provides complete enterprise monitoring functions for Web servers and

database servers

10

FunnelWeb Suite®

—Provides a complete solution set to manage a Web site

•

About the Authors

Bernard Farrell is a seasoned technical writer for Quest Software. Mr. Farrell currently

writes technical documentation for Benchmark Factory. Mr. Farrell's writing

experience centers generally on benchmarking, and specifically on database

applications.

Kevin Dalton, P.E., currently is the Development Manager of Benchmark Factory for

Quest Software. Mr. Dalton's responsibilities focus on the day-to-day development of

the Benchmark Factory application. Mr. Dalton was the initial architect/developer of

the Benchmark Factory Web Module and brings an excess of 12 years of technical

expertise of which five of those years centered on performance and load testing. Prior

to joining the Quest team, Mr. Dalton was a consultant for the Anheuser-Busch

companies.

About Quest Software

Quest Software, Inc. (NASDAQ: QSFT) is the leading provider of application

management solutions. Our software gives businesses confidence that their vital

applications will be available and performing well — while simultaneously driving

down the costs of managing them. By focusing on the people and technology that

make applications run, Quest enables IT professionals to achieve more with fewer

resources and get the most out of existing application investments. Founded in 1987

and based in Irvine, California, Quest Software has offices around the globe and more

than 1,800 employees. Approximately 10,000 businesses worldwide use Quest

Software because

business runs better on Quest

.

For more information on Quest and Benchmark Factory, visit

www.quest.com

.

11