questions on scalability

Thread view:

Enable EMail Alerts

Start New Thread

Thread rating:

Magnus Warker - 14 Apr 2006 07:00 GMT

Dear group,

we are currently building a database cluster with Oracle 10g on two
Primepower 250 machines. Our goal is to start with small hardware
equipment, because of the costs, but to be able to upgrade the equipment
and make it stronger, as the load becomes stronger.

Currently we have two main questions concerning this long-term issue:

1) Measurement of load

How can we define the load with respect to hardware requirements? Or: Which
hardware is required to serve a specific load and how is this load defined,
e. g. number of users, connections and so on?

2) Scaling up

How can we ensure that the hardware equipment is always capable to meet the
requirements? How should we upgrade the hardware? And the most important
thing: What can we do when the PP 250 machine is at its maximum upgradable
limit?

I would like to hear from people that are in a similar situation, or just
talk about these things.

Regards,
Magnus

Reply to this Message

Bob Jones - 16 Apr 2006 18:22 GMT

> Dear group,
>
[quoted text clipped - 12 lines]
> defined,
> e. g. number of users, connections and so on?

You just have to know what your hardware can handle. There is no better way
than testing and experience. The work load is also highly dependent on
applications and data.

> 2) Scaling up
>
[quoted text clipped - 3 lines]
> thing: What can we do when the PP 250 machine is at its maximum upgradable
> limit?

You can add more nodes to the cluster or upgrade existing nodes. I prefer
the latter because more nodes require more maintenance work.

Reply to this Message

Magnus Warker - 19 Apr 2006 04:45 GMT

Dear Bob,

thank you for your comments.

So for scaling up the cluster we have two possibilities:

- upgrade the hardware of single nodes
- add additional nodes to the cluster

I wonder how to decide when to perform which action.

Is this really all one can do or does one have something like a strategy
when planning the cluster?

Regards,
Magnus

>> 1) Measurement of load

> You just have to know what your hardware can handle. There is no better
> way than testing and experience. The work load is also highly dependent on
> applications and data.

>> 2) Scaling up

> You can add more nodes to the cluster or upgrade existing nodes. I prefer
> the latter because more nodes require more maintenance work.

Reply to this Message

Bob Jones - 21 Apr 2006 03:59 GMT

If you have already purchased the hardware, you just have to go with what
you have. There is no way to estimate how much hardware is needed without
knowing your applications. Tuning and monitoring, are perhaps the best
strategy.

> Dear Bob,
>
[quoted text clipped - 24 lines]
>> You can add more nodes to the cluster or upgrade existing nodes. I prefer
>> the latter because more nodes require more maintenance work.

Reply to this Message

NetComrade - 21 Apr 2006 16:27 GMT

>2) Scaling up
>
>How can we ensure that the hardware equipment is always capable to meet the
>requirements? How should we upgrade the hardware? And the most important
>thing: What can we do when the PP 250 machine is at its maximum upgradable
>limit?

If your application is not total crap (you have proper schema design,
you have proper indices, you have tuned sql, you use bind variables,
most of your SQL doesn't do more than 500 logical reads, you do as
much with 1 SQL statement as possible), and you're building an OLTP
type of an application with scalability in mind (read Tom Kyte's
books), 2-4 CPU machines should be able to handle thousands to
hundreds thousands of users concurrently.

-andrey

.......
We run Oracle 9.2.0.6 on RH4 AMD
we are currently looking for a DBA
remove NSPAM to email

Reply to this Message

Bob Jones - 22 Apr 2006 01:23 GMT

>>2) Scaling up
>>
[quoted text clipped - 11 lines]
> books), 2-4 CPU machines should be able to handle thousands to
> hundreds thousands of users concurrently.

Well said, but some OLTP apps are very compute intensive. Without enough
info, I am only willing to bet a cheap watch on the 2-4 CPU estimate.

Reply to this Message

NetComrade - 22 Apr 2006 06:06 GMT

Anything that needs to be computed more then a certain amount of times
per day should require a summary table. You pay your licenses per CPU,
not per GIG of data stored.

But yes.. I said in an almost optimal world 2-4 CPUs will be enough to
support a large amount of users. Testing an application is always
needed prior to any 'real 'deployment. Some vendors are willing to give
you machines to 'test', from others you can 'rent'.

I am fortunate enough to be able to influence the application design at
my current job. And although it's not perfect in many aspects (poor
schema design in places), after seeing some other Oracle installations
I know ours is better than 85% of them (at least in the small to medium
(.5TB) database installations I have seen). At any given time we have
tens of thousands and sometimes hundreds of thousands of people on our
web site going through fairly database intensive pages, all supported
on three quad CPU servers, 2 of which are very underutilized. We cache
some non-user specific data on application servers, we cache some
user-specific data in cookies, we use replication/snapshots to avoid
queries across the link, we use summary tables so that we don't have to
calculate many 'sums' and 'avgs' on the fly (sports statistics
data).This has allowed us to not spend 50% of datacenter budget on
Oracle licenses.

Reply to this Message

Magnus Warker - 24 Apr 2006 18:06 GMT

Dear Andrey,

unfortunately we have no information on the quality of the application's
design, because we are just hosting it.

At the moment we would like to know what are the criteria that influence the
hardware requirements, independent from the application itself. For
example, if we know the number of transactions, what does this mean for
which hardware components, e. g. CPU, RAM, etc. Is there something like a
formula for this?

Regards,
Magnus

> If your application is not total crap (you have proper schema design,
> you have proper indices, you have tuned sql, you use bind variables,
[quoted text clipped - 10 lines]
> we are currently looking for a DBA
> remove NSPAM to email

Reply to this Message

Joel Garry - 24 Apr 2006 19:30 GMT

> Dear Andrey,
>
[quoted text clipped - 6 lines]
> which hardware components, e. g. CPU, RAM, etc. Is there something like a
> formula for this?

Craig Shallahammer has some stuff:
http://resources.orapub.com/category_s/2.htm

When it comes down to it, the application makes the most difference.

jg

Signature

@home.com is bogus.
http://catless.ncl.ac.uk/Risks/24.22.html#subj1

Reply to this Message

Fortuitous Technologies - 28 Apr 2006 19:27 GMT

> Dear group,
>
[quoted text clipped - 8 lines]
>
> How can we define the load with respect to hardware requirements? Or: Which

> 2) Scaling up
>
> How can we ensure that the hardware equipment is always capable to meet the
> requirements?

Magnus,

Your questions are usually best answered by a performance and
capacity planning study. Several firms do this type of work, such as
ours. The best case scenario is to to the study well before you buy any
hardware, so you don't get locked into a ill performing system. Take a
look at some of the articles in
http://www.fortuitous.com/en/resources/whitepapers/

We normally try to analyze the entire system: Application, IO, CPU,
memory, and network interconnects. Measurement of load is done with
several performance related tools, and/or log files. We also do
analysis for the distributed locking managers and caching mechanisms of
Oracle RAC's Cache Fusion for sizing of clusters and grids.

Phil Carinhas,
Fortuitous Technologies, Inc
http://fortuitous.com

Reply to this Message