Home | Contact Us | FAQ | Search & Site Map | Link to Us
Sign In | Join | Other 45 Sites in Network
Home
Discussion Groups
Database Servers
DB2InformixIngresMS SQLOraclePervasive.SQLPostgreSQLProgressSybase
Desktop Databases
FileMakerFoxProMS AccessParadox
General
General DB TopicsDatabase Theory
Related Topics
Java Development.NET DevelopmentVB DevelopmentMore Topics ...

Database Forum / Oracle / Oracle Server / December 2003

Tip: Looking for answers? Try searching our database.

Deadly sins againts database performance/scalability

Thread view: 
Enable EMail Alerts  Start New Thread
Thread rating: 
Dusan Bolek - 25 Nov 2003 16:03 GMT
Hello,

I'm just creating some document how developers should/must not behave
when dealing with Oracle database. I written down some basic rules and
came with idea what's the most stupid thing that you can do to your
database.
I have two on mind, first one is not using BIND variables and the
second one is using COMMIT in a loop statement. However, there will be
more of these.
So I have question to you here. What's the most stupid, but in real
world foundable, mistakes in development applications using Oracle
datatabase?

--
with regards

Dusan Bolek
Norman Dunbar - 25 Nov 2003 16:35 GMT
> Hello,

<SNIP>
> So I have question to you here. What's the most stupid, but in real world
> foundable, mistakes in development applications using Oracle datatabase?

1. Not taking backups :o)
2. Adding indexes all over the place for performance reasons.
3. No (proper) change control procedures.
4. Exporting from the database and defaulting COMPRESS to Y, even worse,
exporting with COMPRESS=Y explicitly.
5. Allowing developers to create obbjects in a tablespace with a
different extent size to the tablespace defaults.
6. Having numerous indexes (on the same table) which could be replaced by
one or two carefully thought out indexes. (you know, indexes on A and B
and A,B and C and A,B,C and B,C ....)
7. Putting the ref int checks in the application instead of the database.
8. Ditto, but keeping business rules in the app instead of the db.
9. Using standalone procs and functions instead of packages.
10. When actually writing packages, not separating the PACKAGE specs from
the PACKAGE BODY so every time you change the code in a body, you
re-create the spec as well and invalidate everything that depends upon it.
11. Using a table to hold a sequence number is a bad one. You know, read
the number and lock it, increment it and write it back - then commit at
some point in the far future. Even committing ASAP is a major problem as
soon as you get more than 5 or so users all doing the same thing.
12. Even worse, having a single table for all the sequence numbers in the
system.
13. Creating 'temporary' tables to hold intermediate results MS SQL Server
style.
14. Converting triggers from another db system into Oracle and discovering
that the other system allows you to update rows in the table that has
triggered the trigger leading to mutating table errors on Oracle (which
IMHO does it correctly !)
15. Not enough SQL reuse.

I'm sure that there are more :o)

Cheers,
Norm.

Signature

Delete the obvious bit from my email address to reply by email.

Norman Dunbar - 26 Nov 2003 08:51 GMT
<BIG SNIP>

Sorry folks, looks like my formatting went the same way as Australias
chance of winning the RWC - it all went wrong at the end !

Cheers,
Norm.

Signature

Delete the obvious bit from my email address to reply by email.

Niall Litchfield - 25 Nov 2003 16:49 GMT
> Hello,
>
[quoted text clipped - 8 lines]
> world foundable, mistakes in development applications using Oracle
> datatabase?

No primary key - rare
No foreign keys - common
No bind variables
Create intermediate tables as often as possible and then drop them when done
with.
Use PL/SQL instead of SQL
No Bind variable.
Use Java SP instead of PL/SQL
Use EJB instead of the Database.
Use Net instead of the Database
No bind variables   .
Avoid array fetch at all costs.
Parse repeatedly.
Test on your laptop with a maximum of 5 rows per table.
Don't test at all, that is what pilots are for.
nO binD variableS.
Return all of the table to your client and filter on the client to avoid
loading the server.
Never document anything - noone will read it.
Never have a database model - that is just for classes.
no bind variables

:(

Signature

Niall Litchfield
Oracle DBA
Audit Commission UK

Daniel Morgan - 25 Nov 2003 20:44 GMT
>>Hello,
>>
[quoted text clipped - 32 lines]
>
> :(

You forgot No Bind Variables.  ;-)

Signature

Daniel Morgan
http://www.outreach.washington.edu/ext/certificates/oad/oad_crs.asp
http://www.outreach.washington.edu/ext/certificates/aoa/aoa_crs.asp
damorgan@x.washington.edu
(replace 'x' with a 'u' to reply)

Jeff - 26 Nov 2003 18:15 GMT
What are bind variables?  

;-)

>> Hello,
>>
[quoted text clipped - 32 lines]
>
>:(
Norman Dunbar - 27 Nov 2003 08:52 GMT
> What are bind variables?
>
> ;-)

Hi Jeff,

who mentioned bind variables then. I don't think it was Niall.
:o)

Cheers,
Norm.

Signature

Delete the obvious bit from my email address to reply by email.

Andy Spaven - 27 Nov 2003 15:52 GMT
Hi all

I want to laugh about the bind variables posts but I might cry given my
current employers attitude - All I can say is thank god for
cursor_sharing=force :-))

Andy

> > What are bind variables?
> >
[quoted text clipped - 7 lines]
> Cheers,
> Norm.
Niall Litchfield - 27 Nov 2003 16:10 GMT
Actually I'm beginning to regret that post now.

As Joel rightly says there is a real danger in creating a list like that
one. In my case my post might just have suggested that one should always use
bind variables (I believe you could read it that way :) ) and this is of
course untrue, one should always use them, except where they are
inappropriate.

Signature

Niall Litchfield
Oracle DBA
Audit Commission UK

>
> > What are bind variables?
[quoted text clipped - 8 lines]
> Cheers,
> Norm.
Daniel Morgan - 27 Nov 2003 18:28 GMT
> Actually I'm beginning to regret that post now.
>
[quoted text clipped - 3 lines]
> course untrue, one should always use them, except where they are
> inappropriate.

And they are in appropriate ... when?

Signature

Daniel Morgan
http://www.outreach.washington.edu/ext/certificates/oad/oad_crs.asp
http://www.outreach.washington.edu/ext/certificates/aoa/aoa_crs.asp
damorgan@x.washington.edu
(replace 'x' with a 'u' to reply)

Niall Litchfield - 28 Nov 2003 08:43 GMT
Where you have skewed data would be the classic example.

select employee_name,salary
from emp
where dept='FACTORY FLOOR';

might be expected to run optimally with a FTS.

select employee_name,salary
from emp
where dept='CEO OFFICE';

probably would best use an index scan. Assuming a reasonable proportion of
workers to chief execs :(

if you issue

select employee_name,salary
from emp
where dept=:b1;

which are you going to want?

Signature

Niall Litchfield
Oracle DBA
Audit Commission UK

--
Niall Litchfield
Oracle DBA
Audit Commission UK

>
> > Actually I'm beginning to regret that post now.
[quoted text clipped - 6 lines]
>
> And they are in appropriate ... when?
Jonathan Lewis - 29 Nov 2003 08:52 GMT
Answer in-line

Signature

Regards

Jonathan Lewis
http://www.jlcomp.demon.co.uk

 The educated person is not the person
 who can answer the questions, but the
 person who can question the answers -- T. Schick Jr

One-day tutorials:
http://www.jlcomp.demon.co.uk/tutorial.html

Three-day seminar:
see http://www.jlcomp.demon.co.uk/seminar.html
____UK___November

The Co-operative Oracle Users' FAQ
http://www.jlcomp.demon.co.uk/faq/ind_faq.html

> > Actually I'm beginning to regret that post now.
> >
[quoted text clipped - 5 lines]
>
> And they are in appropriate ... when?

DSS systems where the queries are few and
large, and the optimiser's need for precise inputs
is more important than a (relatively) minor increase
in CPU and latch costs on parse/optimise.
Ryan Gaffuri - 01 Dec 2003 18:57 GMT
> Answer in-line
>
[quoted text clipped - 13 lines]
> is more important than a (relatively) minor increase
> in CPU and latch costs on parse/optimise.

I also disgree that you should never create intermediate tables and
drop them. I think this is taken to an extreme. Now in a high
transaction system you should not because it causes latch contention
and affects throughput.

However, in a DSS system or for nightly batch processes, Ive found
VERY large improvements in performance by doing create table as
nologging in paralel. Its much faster than inserting into a global
temp table. you dont have to worry about throughput if your looking at
5-10 users on the system max. Now you need to document it well(which
most people dont do), so people know what to alter if you need to
scale to more users. If I only have a certain window of time to get a
series of batch processes done, Ill use temp tables like this. I find
they are very useful when working with remote databases. You do a
quick create table as to get just the data you need, then do your
joins to the local table, so your not pushing data across a DB link
for 10 different queries.

However, this will not scale to alot of users. So it needs to be well
documented.
Daniel Morgan - 02 Dec 2003 01:26 GMT
>>Answer in-line
>>
[quoted text clipped - 36 lines]
> However, this will not scale to alot of users. So it needs to be well
> documented.

If you ever want to have some fun do the following:

CREATE TABLE test (testcol VARCHAR2(20)) NOLOGGING;
ALTER SESSION SET EVENTS '10046 trace name context forever, level 12';
INSERT INTO test VALUES ('ABC');
DROP TABLE test;
ALTER SESSION SET sql_trace = FALSE;

Then ...

CREATE GLOBAL TEMPORARY TABLE test (testcol VARCHAR2(20))
ON COMMIT DELETE ROWS;

ALTER SESSION SET EVENTS '10046 trace name context forever, level 12';
INSERT INTO test VALUES ('ABC');
COMMIT;
ALTER SESSION SET sql_trace = FALSE;

Run the trace files through TKPROF.

Take a good look at what Oracle does to drop that table. It is not
exactly a thing of beauty. Necessary ... veryt expensive.

Signature

Daniel Morgan
http://www.outreach.washington.edu/ext/certificates/oad/oad_crs.asp
http://www.outreach.washington.edu/ext/certificates/aoa/aoa_crs.asp
damorgan@x.washington.edu
(replace 'x' with a 'u' to reply)

Ryan Gaffuri - 02 Dec 2003 18:08 GMT
> >>Answer in-line
> >>
[quoted text clipped - 59 lines]
> Take a good look at what Oracle does to drop that table. It is not
> exactly a thing of beauty. Necessary ... veryt expensive.

i know what it does. Ive traced it. Ive also noticed that what it does
causes contention for resources and affects throughput. When doing
batch processing your often not concerned with throughput. Your
concerned with response time. That resource contention is not much of
a concern when you do not have to worry about alot of users. when your
primary concern is getting the batch done quickly.

Also, alot of people(including us) are moving to shared storage where
you run multiple servers on one storage system. In this case physical
I/Os become important. Since you may have alot of servers hitting one
big pipe. I found that create table as nologging significantly reduces
physical i/o over inserting into a global temporary table.

I have found that tuning for throughput and tuning for response time
can lead to somewhat different paths in some circumstances.

as I have said before this process will scale up in terms of more
data, etc... involved but will not scale for many, many concurrent
users. If you need to balance batch processes and concurrency then
this would not be appropriate.
Daniel Morgan - 02 Dec 2003 20:26 GMT
>>>>Answer in-line
>>>>
[quoted text clipped - 80 lines]
> users. If you need to balance batch processes and concurrency then
> this would not be appropriate.

And you are, I presume, using the APPEND hint.
Signature

Daniel Morgan
http://www.outreach.washington.edu/ext/certificates/oad/oad_crs.asp
http://www.outreach.washington.edu/ext/certificates/aoa/aoa_crs.asp
damorgan@x.washington.edu
(replace 'x' with a 'u' to reply)

Vladimir Andreev - 08 Dec 2003 12:18 GMT
> And they are in appropriate ... when?

-- When they hide important information from the CBO:
* low distinct cardinality columns with uneven distribution of the
frequencies.
In such cases, you create histograms and, ideally, use literals in
predicates. That's somehow addressed by Oracle since 9.0.2 with the
bind variable peeking feature, but still... it only has an effect on
the hard parse, since binding a different value does not invalidate
the parsed cursor (that's the whole point...)

* when using list or range partitioning
Using literals allows the CBO to prune partitions at parse time,
rather than at execute time

Cheers,
Flado
Joe Weinstein - 25 Nov 2003 17:30 GMT
> Hello,
> I'm just creating some document how developers should/must not behave
[quoted text clipped - 7 lines]
> world foundable, mistakes in development applications using Oracle
> datatabase?

The number one performance rule for Oracle or any other RDBMS is
"Don't Move the Data". The biggest sin is treating the DBMS as a
dumb filing system, and pulling huge wads of raw data out into a
client to massage and return row-by-row or digest into a report.
DBMSes are designed for efficient set-based processing. Try to
extract only the data needed to display. For any data extracted that
is non-volatile or under the total control of the one client application,
to cache retrieved data in a long-running client (or middle tier) to
avoid burdening the DBMS with repeated identical queries.
   Do all the heavy big-volume processing where the data is, in
the DBMS with procedures. You build your sawmill where the trees are.
   I know of one big application vendor whose payroll program is
a straight port of an ISAM app to RDBMS. It takes 8 hours on big
iron to 'balance the checkbook' of 40,000 employees. If this were
done in procedures, it'd take minutes. I know of another big app
vendor whose client interface has about 250 separate business methods.
A typical user would do his/her work calling these methods about 50-100
times a day. More than 80% of these methods have processing that
includes a fresh query to the DBMS to return a list of all the countries
in Europe. I understand the political volatility in Europe has been higher
recently... ;-)

Joe Weinstein at BEA

> --
> with regards
>
> Dusan Bolek
Hans Forbrich - 25 Nov 2003 19:12 GMT
> Hello,
>
[quoted text clipped - 8 lines]
> world foundable, mistakes in development applications using Oracle
> datatabase?

Aside from the traditional onces (appropriate SGA sizing, BIND
variables, etc.), some negative impact development
philosophy/implementation items that I've observed:

1) Attempt to do the security in application
- critical hole when multiple access techniques are allowed
- requires duplication of the s/w
- usually not as efficient as using Oracle's 'row level security'

1a) Not using Oracle's Resource manager
   
2) Attempt to do ALL application logic in front/mid tier
- invariably moves excess amounts of data
- move data should be restricted to minimum resolved set so use joins
and where clause properly

3) Using a procedural language (C, C++, PL/SQL, VB) when the same
results could be obtained by one (or more) well-thought-out SQL
statement(s)
- usually a procedural language involves stepping through a cursor
- if a cursor loop ONLY involves data transformation, chances are high
that it could be accomplished by a single SQL statement with appropriate
selectivity

4) Using TEMP tables
- There are valid reasons for temp tables;
- However, several uses of temp tables are simply a workaround to
"advanced" isolation level capabilities found in some other RDBMSs.
Oracle generally does not require such a workaround as it ensures that
the data in a transaction can not be seen outside the transaction's
session.

5) IMO, every index should be reviewed every 6 months toi determine
whether it's needed
- Too often have I seen indexes created to meet 1-shot or limited use
requirements and left around;
-  Remove unneeded or unnecessary (or duplicate) indexes!

6) Reinventing the wheel.  
- I've seen a lot of organizations literally spend millions of dollars
duplicating built-in functionality of (intermedia's text search,
workflow, workspace management/long transaction, advanced queueing,
htp/htf, advanced grouping functions, spatial/location management,
etc.);
- This not only wastes development effort and costs extra maintenance,
it dilutes the value of the product, making it appear more expensive
than it is;
- Developers should know and understand Oracle's capabilities;
- Reinventing the wheel invariably violates 1) 2) and 3) above.

7) Insisting in yet another instance on that machine, 'cause each
application should have it's own instance
- likely can't be tuned
- wastes license $ due to semaphore, shmem and CPU overhead for each
instance

8) Insisting that the machine should host ALL of the database instance,
the access tier, the report generator, etc.  
- likely can't be tuned
- wastes license $ (eg: need 4 CPU to do 2 CPU's worth of database work)

9) Using RBO; not running appropriate stats
- In 9i, CBO is quite acceptable AND will take into account the changes
in environment.

10) Not using appropriate index types (eg: Function Based indexes)

11) Not allowing the DBA to actually administer the DB
- Some organizations don't get apprpriate DBA skills or allow developers
to override DBA suggestions

12) Not evaluating or using appropriate options
- Partitioning can and does provide significant performance and
scalability benefits and is relatively easy to cost justify if the
organization has any sort of 'business case justification' process.

13) Blindly following rules of thumb without understanding them.
Daniel Morgan - 25 Nov 2003 21:43 GMT
> Hello,
>
[quoted text clipped - 13 lines]
>
> Dusan Bolek

Having read everyone else's list.

1. Developers that don't understand the implications of the multiversion
concurrency model.

2. Developers that don't understand the implications of reads not
blocking writes and writes not blocking reads.

3. DBAs that don't understand the implications of ORA-01555.

4. Those schooled in other RDBMS products that insist on treating Oracle
as if were the other product.

5. Organizations that don't conduct code reviews.

6. Developers of testing prorams that can't read code and don't know how
to identify potential weaknesses and test for them.

Signature

Daniel Morgan
http://www.outreach.washington.edu/ext/certificates/oad/oad_crs.asp
http://www.outreach.washington.edu/ext/certificates/aoa/aoa_crs.asp
damorgan@x.washington.edu
(replace 'x' with a 'u' to reply)

Daniel Roy - 25 Nov 2003 22:14 GMT
Norman did a great job listing the database development sins, but I
have 1 more: Doing everything that Siebel enforces its customers to
do! For example:
- Enforce the use of RBO instead of CBO!
- Rebuild indexes periodically, in the hope that "it will help
performance"
- create gazillions of indices on an OLTP system (it's not common in
Siebel that more than 100 indices are defined on a table)
- Have no reference integrity at the database level!
- Avoid at all costs to use MV's or views!!!! (they prefer instead to
use giant SQL statements).

Daniel Roy
Siebel Developer!

> Hello,
>
[quoted text clipped - 8 lines]
> world foundable, mistakes in development applications using Oracle
> datatabase?
Anurag Varma - 25 Nov 2003 22:28 GMT
> Norman did a great job listing the database development sins, but I
> have 1 more: Doing everything that Siebel enforces its customers to
[quoted text clipped - 10 lines]
> Daniel Roy
> Siebel Developer!

--snip--

he he .. and you would wonder how Tom Siebel who was former group vice president for Oracle USA
could be incharge of an application which breaks so many rules in the Oracle world.

:) Maybe getting back at Larry and his customers ...

Anurag
mcstock - 25 Nov 2003 23:08 GMT
| > Norman did a great job listing the database development sins, but I
| > have 1 more: Doing everything that Siebel enforces its customers to
[quoted text clipped - 19 lines]
|
| Anurag

well, after having spent the better part of the last 15 years cleaning up
after Oracle consultants and untraining bad habits taught by Oracle
Education, my experience would say that maybe it's just a matter of
consistency

(of course, as an xoraclite, my observations may be tainted ;-{)

--mcs
Daniel Morgan - 25 Nov 2003 23:42 GMT
>>Norman did a great job listing the database development sins, but I
>>have 1 more: Doing everything that Siebel enforces its customers to
[quoted text clipped - 19 lines]
>
> Anurag

If Larry doesn't get PeopleSoft he may turn his attentions elsewhere. Hmmmm!

Signature

Daniel Morgan
http://www.outreach.washington.edu/ext/certificates/oad/oad_crs.asp
http://www.outreach.washington.edu/ext/certificates/aoa/aoa_crs.asp
damorgan@x.washington.edu
(replace 'x' with a 'u' to reply)

Hans Forbrich - 25 Nov 2003 23:51 GMT
> If Larry doesn't get PeopleSoft he may turn his attentions elsewhere. Hmmmm!

I believe he's on record saying that Peoplesoft is the first of a set.
Others have been identified as 'possible when the price is right'.  

(IIRC, taken from an interview in Internet Week, May/June timeframe -
roughly the same weekas the original bid for PeopleSoft.)
Joel Garry - 25 Nov 2003 23:48 GMT
> Hello,
>
[quoted text clipped - 8 lines]
> world foundable, mistakes in development applications using Oracle
> datatabase?

Letting developers into the production database.

jg
--
@home.com is bogus.
http://www.gripe2ed.com/scoop/story/2003/11/24/02356/143
Galen Boyer - 28 Nov 2003 00:48 GMT
> Letting developers into the production database.

Having DBA's make decisions about an application's database when
they haven't been involved in the datamodeling or coding effort.

Signature

Galen Boyer

Hans Forbrich - 28 Nov 2003 09:12 GMT
> > Letting developers into the production database.
>
> Having DBA's make decisions about an application's database when
> they haven't been involved in the datamodeling or coding effort.

Avoiding DBA (or at least data architect) input during design.  

Having developers with no datamodeling experience dictate table and
column definitions.
Galen Boyer - 28 Nov 2003 12:34 GMT
>> > Letting developers into the production database.
>>
[quoted text clipped - 3 lines]
>
> Avoiding DBA (or at least data architect) input during design.

That's the point.  There are too many places where the DBA's
aren't involved in anything but locking down the database.  They
shouldn't even be involved in that if that's all they do.

> Having developers with no datamodeling experience dictate table
> and column definitions.

Having DBA's that no nothing about the application development
dictate coding policies.

Signature

Galen Boyer

Daniel Morgan - 28 Nov 2003 18:37 GMT
>>>Letting developers into the production database.
>>
[quoted text clipped - 5 lines]
> Having developers with no datamodeling experience dictate table and
> column definitions.

A number of you have hit this point from various angles and I would like
to try to merge them all and thus throw gasoline on the fire.

1. There are DBAs that think their job consists of backup-recovery,
creating users, and denying access to everything because end-users only
create problems. I would cateogize them, for the most part, as DBAs that
couldn't write and debug a stored procedure if you threatened them with
a sharp object.

2. There are DBAs that are also competent developers having come up
through the ranks and are just sick and tired of developers that throw
junk over the cubicle wall, don't run explain plan, don't use bind
variables, and haven't a clue of the implications of what they write,
and whine like babies when the phrase "code review" is mentioned.

3. There are developers that are competent and can't understand why DBAs
stand in their way trying to get the job done.

4. There are mnagers that don't have the skill set required to define
projects, hire good people, fire bad people, and train those that need
more skill to do their job properly.

Tackle number 4 and the other three resolve themselves quickly.

Signature

Daniel Morgan
http://www.outreach.washington.edu/ext/certificates/oad/oad_crs.asp
http://www.outreach.washington.edu/ext/certificates/aoa/aoa_crs.asp
damorgan@x.washington.edu
(replace 'x' with a 'u' to reply)

Jeff - 01 Dec 2003 14:32 GMT
>2. There are DBAs that are also competent developers having come up
>through the ranks and are just sick and tired of developers that throw
>junk over the cubicle wall, don't run explain plan, don't use bind
>variables, and haven't a clue of the implications of what they write,
>and whine like babies when the phrase "code review" is mentioned.

Do you write for Dilbert?  :-)

>4. There are mnagers that don't have the skill set required to define
>projects, hire good people, fire bad people, and train those that need
>more skill to do their job properly.
>
>Tackle number 4 and the other three resolve themselves quickly.

Abso-frikkin-lutely!
Jeff - 01 Dec 2003 14:28 GMT
>> > Letting developers into the production database.
>>
[quoted text clipped - 5 lines]
>Having developers with no datamodeling experience dictate table and
>column definitions.

Having developers with no datamodeling experience.
Jeff - 01 Dec 2003 14:27 GMT
>> Letting developers into the production database.
>
>Having DBA's make decisions about an application's database when
>they haven't been involved in the datamodeling or coding effort.

LOL.
Ron Reidy - 26 Nov 2003 02:26 GMT
1.  Developers that can't/won't read the docs.
2.  Developers that can't/won't take advantage of Oracle strengths
(black box phenomona).
3.  Lack of bind vars; lack of reusable SQL; too much LIOs.
4.  Overuse of hints w/o understanding the CBO and how it works.
5.  Lack of budget to get the proper resources (people and hardware).
6.  Propogation of "mythical cure-alls" for performance issues.
7.  Badly architected software (see #3).
8.  The "internet-time" syndrome vs. carefull, well thought out analysis.

Lot's more, I'm sure.

> Hello,
>
[quoted text clipped - 13 lines]
>
> Dusan Bolek

Signature

Ron Reidy
Oracle DBA

Connor McDonald - 26 Nov 2003 12:23 GMT
> Hello,
>
[quoted text clipped - 13 lines]
>
> Dusan Bolek

Only one...

People on a project who aren't
- passionate about what they do.  
- keen to learn

Everything else falls into place if you can satisfy the above
Daniel Morgan - 26 Nov 2003 17:08 GMT
>>Hello,
>>
[quoted text clipped - 21 lines]
>
> Everything else falls into place if you can satisfy the above

I'll add one more ...

Developers that seem incapable of understanding the basics of
normalization and data integrity.

Again, today, I've been called into look at a database in which it is
impossible to connect related events.

Example:
1. Cases have a case number
2. Citations have a case number and a citation number
3. Fees paid for citations have a case number and a payment number

If you can't see what's wrong ... you're part of the problem.   ;-)
Signature

Daniel Morgan
http://www.outreach.washington.edu/ext/certificates/oad/oad_crs.asp
http://www.outreach.washington.edu/ext/certificates/aoa/aoa_crs.asp
damorgan@x.washington.edu
(replace 'x' with a 'u' to reply)

Billy Verreynne - 27 Nov 2003 06:42 GMT
> Developers that seem incapable of understanding the basics of
> normalization and data integrity.

Amen to that.

> Again, today, I've been called into look at a database in which it is
> impossible to connect related events.
[quoted text clipped - 3 lines]
> 2. Citations have a case number and a citation number
> 3. Fees paid for citations have a case number and a payment number

Almost the exact same problem here (dealing with billing issues and tariffs).

I'm still amazed at the "logic" employed to design relationships like that.

> If you can't see what's wrong ... you're part of the problem.   ;-)

Yeah.. and requires liberal application of the old lead pipe to correct the problem.

--
Billy
ctcgag@hotmail.com - 27 Nov 2003 20:10 GMT
> Again, today, I've been called into look at a database in which it is
> impossible to connect related events.
[quoted text clipped - 5 lines]
>
> If you can't see what's wrong ... you're part of the problem.   ;-)

I'm not sure if I'm part of the problem or not.  If the case-citation
relationship is one-to-one but optional, I don't see the problem.  If it's
one to many, I do (unless you say that money is fungible and it doesn't
matter what the payments are for as long as it adds up to the correct
amount)

Xho

Signature

-------------------- http://NewsReader.Com/ --------------------
Usenet Newsgroup Service              New Rate! $9.95/Month 50GB

Daniel Morgan - 28 Nov 2003 00:14 GMT
>>Again, today, I've been called into look at a database in which it is
>>impossible to connect related events.
[quoted text clipped - 13 lines]
>
> Xho

You may not be part of the problem but you are a strong candidate.  ;-)

With the above you can link citations to cases.
And you can link fees to cases.
But you can not tell which fee goes with which citation.

The following simple change would provide referential integrity.

1. Cases have a case number
2. Citations have a case number and a citation number
3. Fees paid for citations have a citation number and a payment number.

The fees are linked to the case by means of the citation number
providing a complete referential join.

This is a critical skill. And one I see all too rarely.

Signature

Daniel Morgan
http://www.outreach.washington.edu/ext/certificates/oad/oad_crs.asp
http://www.outreach.washington.edu/ext/certificates/aoa/aoa_crs.asp
damorgan@x.washington.edu
(replace 'x' with a 'u' to reply)

Jeff - 01 Dec 2003 14:13 GMT
>This is a critical skill. And one I see all too rarely.

That's very sad for you then, because that skill seems extremely
fundamental--any database developer worth a shatt, should know that stuff
backwards and forwards.
Mark D Powell - 26 Nov 2003 18:44 GMT
> Hello,
>
[quoted text clipped - 8 lines]
> world foundable, mistakes in development applications using Oracle
> datatabase?

Dusan, I would suggest you change your wording about issuing commit
within a loop since in many cases the commits need to be performed
within a loop.  I think it is the frequency of commits rather than the
logical coding structure that is the issue.  In my opinion how
frequently a commit should be issued is dependent on how likely
another session will need update access to the rows being processed.
In a transaction driven system the inventory rows may well require
commits be issued on a per-row or very few row basis.  While another
program in the application, which is the only source of update
activity for a table that has low select activity, would best be
served by one or very few commits.

Be careful in the process of trying to get something into the hands of
developers that you do not over simplify the problem and end up making
a statement that you later consider "dumb".  Writing for developers is
not an easy task.

Good luck
-- Mark D Powell --
Frank - 26 Nov 2003 21:38 GMT
>>Hello,
>>
[quoted text clipped - 28 lines]
> Good luck
> -- Mark D Powell --

I'm sure Dusan referred to the "we have a possible 1555, so let's
commit every ...uhm, 5000 processed rows".
Committing within a loop to "prevent" 1555's from happening is
creating the 1555's!
Signature

Regards, Frank van Bortel

ctcgag@hotmail.com - 30 Nov 2003 00:30 GMT
> >>Hello,
> >>
[quoted text clipped - 31 lines]
> I'm sure Dusan referred to the "we have a possible 1555, so let's
> commit every ...uhm, 5000 processed rows".

But I think this just demonstrates Mark's point.  If you have to already
know what he is talking about in order to know what he is talking about,
then it isn't a very good educational document.

However, your supposition seems to require that developer are 1) reading
the manuals and thinking carefully about what problems may arise, 2) taking
pre-emptive action to forestall those problems, and 3) making things worse
rather than better.

Now 3) I buy completely, but the first two seem foreign to most of my
experience.

> Committing within a loop to "prevent" 1555's from happening is
> creating the 1555's!

I recall first encountering 1555s, and they were definately not created by
committing inside a loop, because I wasn't doing that.  In fact, the
solution to the 1555s did involve moving the commit from outside the loop
to inside a loop, but not doing it in the naive manner I first tried.  (And
it also solved the companion problem, of locking others out of the rows
for too long).

So if you tell people that committing inside the loop is causing the 1555s,
when those existed even before they moved the commit into the loop, it
isn't going to inspire developers to take you seriously.

Xho

Signature

-------------------- http://NewsReader.Com/ --------------------
Usenet Newsgroup Service              New Rate! $9.95/Month 50GB

Frank - 30 Nov 2003 16:47 GMT
>>>>Hello,
>>>>
[quoted text clipped - 59 lines]
>
> Xho

I did not say the opposite was true; having 1555's is not a sure
sign of committing over loops (althought I tend to go look
for them!!!), building them into a loop is a sure recipe for
getting them - sooner or later.

Signature

Regards, Frank van Bortel

Daniel Morgan - 26 Nov 2003 21:55 GMT
<snipped>

> Dusan, I would suggest you change your wording about issuing commit
> within a loop since in many cases the commits need to be performed
> within a loop.  I think it is the frequency of commits rather than the
> logical coding structure that is the issue.
><snipped>

Going to take issue with you here Mark. Granted that nothing should ever
been done dogmatically. But 98% of the time a commit in a loop is not
needed, not desirable, the result of someone moving from another RDBMS
or someone that has poorly configured rollback.

The rare times that a commit in a loop is required are the extreme
exception.

Signature

Daniel Morgan
http://www.outreach.washington.edu/ext/certificates/oad/oad_crs.asp
http://www.outreach.washington.edu/ext/certificates/aoa/aoa_crs.asp
damorgan@x.washington.edu
(replace 'x' with a 'u' to reply)

Mark D Powell - 29 Nov 2003 22:29 GMT
> <snipped>
> >
[quoted text clipped - 11 lines]
> The rare times that a commit in a loop is required are the extreme
> exception.

Daniel, in the real world I would hazard to claim that most updates
are done inside a loop because the majority of DML activity takes
place in batch processing.  It might be online batch or traditional
batch but the programs have all have the same basic feature.  The
programs all perform their logic until there are no more input records
whether those records come from a batch file or a driving cursor.  In
other words the programs all loop.

When the target rows are also the target of concurrent DML access then
commits within a loop are a necessity because you cannot hold rows
that are the target of OLTP activity for several seconds without
adversely impacting performance.  So the question is not whether you
commit in a loop, but how frequently.

Snapshots too old are generally only a problem if the source and
update target are the same table.  When that is the case resorting to
use of a driving table or similar technique will normally avoid the
problem.

Most any shop that runs MRP II, or ERP, which is nearly every major
corporation around has an environment that makes heavy use of batch
syte processes.  It may be a customer who triggers the job but the
task processes the same logic for every target row in a loop.  When
you add the OLTP updates up and compare them to the number of rows
processed via batch the majority of updates come from batch.  And
batch programs by nature involve a driving loop, hence my contention
that it isn't doing updates in a loop structure that is the problem
but rather the frequency of commit.

You can disagree, but I think your are failing to consider how much
real world work is done in batch type processing.

-- Mark D Powell --
Daniel Morgan - 29 Nov 2003 23:53 GMT
> You can disagree, but I think your are failing to consider how much
> real world work is done in batch type processing.
>
> -- Mark D Powell --

And I will. I remember the first time I was a Boeing and wrote code that
did that. Seemed reasonable at the time as I was bringing in gigabytes
every weekend. But now with UNDO I'd argue just the opposite. Disk is so
inexpensive it makes no sense not to just assign the equivalent of 1 or
more drives, often 40+GB to UNDO and let it run to comletion.

But your point is valid in a datawarehouse situation. Can we at least
agree that commits in OLTP loops is almost undoubtedly without merit.
Signature

Daniel Morgan
http://www.outreach.washington.edu/ext/certificates/oad/oad_crs.asp
http://www.outreach.washington.edu/ext/certificates/aoa/aoa_crs.asp
damorgan@x.washington.edu
(replace 'x' with a 'u' to reply)

Howard J. Rogers - 30 Nov 2003 02:46 GMT
> > You can disagree, but I think your are failing to consider how much
> > real world work is done in batch type processing.
[quoted text clipped - 5 lines]
> every weekend. But now with UNDO I'd argue just the opposite. Disk is so
> inexpensive

As one who frequently quotes the 'disk space is cheap' mantra, just for
laughs, let me state that "proper" disk space is *not* cheap... EMC aren't
exactly bargain basement material.

Furthermore, RAM is definitely not cheap, and the more disk space you have,
the more blocks there are which need a home in the buffer cache.

So I'm not agreeing or disagreeing with you... just saying things aren't as
simple as that, and there are subtleties to be considered.

>it makes no sense not to just assign the equivalent of 1 or
> more drives, often 40+GB to UNDO and let it run to comletion.

Bear in mind the Oracle automatic undo algorithm: the more space you assign
to the undo tablespace, the more undo segments you end up acquiring. The
more undo segments, the more undo segment header blocks. The more undo
segment header blocks, the more Oracle overhead there is to manage the
thing. And the more undo blocks in general, the bigger your buffer cache had
better be.

The "sense" in automatic undo is that left to its own devices (ie,
implemented poorly) it can consume resources at a rate of knots.

Regards
HJR

> But your point is valid in a datawarehouse situation. Can we at least
> agree that commits in OLTP loops is almost undoubtedly without merit.
Daniel Morgan - 30 Nov 2003 18:42 GMT
>>>You can disagree, but I think your are failing to consider how much
>>>real world work is done in batch type processing.
[quoted text clipped - 28 lines]
> The "sense" in automatic undo is that left to its own devices (ie,
> implemented poorly) it can consume resources at a rate of knots.

A fascinating phrase. Please explain "rate of knots." It is new to me.

I'm not writing some new nonsense mythology that goes "NEVER commit in a
loop." I'm meerly arguing that it should be done if required. And "if
required" means you've tried it without and can't make it work. Lets
remember that almost every post on UNDO, including yours, have advocated
pushing the undo retention out as far as possible to enable using
DBMS_FLASHBACK for recovery.

In the end ... the compromise to be made ... is between the cost of
hardware and the cost of an ORA-01555. I'll continue to argue that the
hardware is far less expensive in almost every situation.
Signature

Daniel Morgan
http://www.outreach.washington.edu/ext/certificates/oad/oad_crs.asp
http://www.outreach.washington.edu/ext/certificates/aoa/aoa_crs.asp
damorgan@x.washington.edu
(replace 'x' with a 'u' to reply)

Howard J. Rogers - 30 Nov 2003 19:16 GMT
> > As one who frequently quotes the 'disk space is cheap' mantra, just for
> > laughs, let me state that "proper" disk space is *not* cheap... EMC aren't
[quoted text clipped - 20 lines]
>
> A fascinating phrase. Please explain "rate of knots." It is new to me.

Speedily.

> I'm not writing some new nonsense mythology that goes "NEVER commit in a
> loop." I'm meerly arguing that it should be done if required. And "if
> required" means you've tried it without and can't make it work. Lets
> remember that almost every post on UNDO, including yours, have advocated
> pushing the undo retention out as far as possible to enable using
> DBMS_FLASHBACK for recovery.

That's actually not true. I did, for a time, recommend huge undo tablespaces
precisely because I liked the idea of being able to flashback almost as far
in the past as you could ever want to. But I quickly stopped suggesting that
when Jonathan (I think) pointed out the impact of vast amounts of empty undo
space on the 'allocate this new transaction to a new undo segment'
algorithm. A bit of testing confirmed that I had just made my biggest-ever
mistake in understanding Oracle. For the past year and more, I have
therefore been saying that automatic undo needs to be set rather carefully,
because otherwise you will swamp your buffer cache with Oracle overhead (ie
undo segment headers for starters). Obviously, there is a tension between
'minimal but adequate' and 'I'd like to do flashback' (which is one reason
why I think you'll find Jonathan doesn't like flashback very much). I
actually like flashback a lot, and would arrange things accordingly... but
like everything good in Oracle, it comes with a cost, and that cost needs to
be monitored and tuned for.

> In the end ... the compromise to be made ... is between the cost of
> hardware and the cost of an ORA-01555. I'll continue to argue that the
> hardware is far less expensive in almost every situation.

But ORA-1555 can be solved with a much more modest and realistic setting of
undo_retention than you suggest I've been recommending in the past.... how
long is your longest query (answerable from v$undostat) should govern
undo_retention. Not "push it out as far as possible" or throw entire hard
disks at it.

In short, over-doing the undo tablespace can limit scalability and
performance because you can kiss goodbye to rather a lot of your buffer
cache in the process. And I've been consistent in saying that for a very,
very long time.

Regards
HJR
Daniel Morgan - 30 Nov 2003 21:47 GMT
Comments in-line.

>>>As one who frequently quotes the 'disk space is cheap' mantra, just for
>>>laughs, let me state that "proper" disk space is *not* cheap... EMC
[quoted text clipped - 60 lines]
> like everything good in Oracle, it comes with a cost, and that cost needs to
> be monitored and tuned for.

I apparently missed the change. A year ago you say. Hmmm. Well I've been
 building most OLTP systems with 24 hour undo retention and have yet to
find a real-world penalty. Thanks for the heads-up. I will keep my eyes
open a bit more.

>>In the end ... the compromise to be made ... is between the cost of
>>hardware and the cost of an ORA-01555. I'll continue to argue that the
[quoted text clipped - 5 lines]
> undo_retention. Not "push it out as far as possible" or throw entire hard
> disks at it.

My use of undo retention at 24 hours has been based on the backup cycle
rather than specifically addressing ORA-01555 which usually requires not
more than 30 minutes.

> In short, over-doing the undo tablespace can limit scalability and
> performance because you can kiss goodbye to rather a lot of your buffer
[quoted text clipped - 3 lines]
> Regards
> HJR

Not doubting you in any manner I went back to the Google archives and
found this:

===================================
From: Howard J. Rogers (howardjr2000@yahoo.com.au)
Subject: Re: How to calculate the optimal size for undo segments
View: Complete Thread (6 articles)
Original Format
Newsgroups: comp.databases.oracle.server
Date: 2003-03-07 04:17:01 PST

Of course: you can't measure these things until you've already got an
undo tablespace, so what you are supposed to do is to create a massive
tablespace that you reckon will contain the worst case you can possible
imagine, and let your users do their thing.... that populates
v$undostat, and you take it from there, sizing the tablespace down or up
as the case may be.
===================================

It was in that thread that you wrote:

"So there's a balance to be struck between oodles of undo space and a
working buffer cache."

So being a Yank I went for "massive" as I didn't understand what an
oodle is. Thanks for the clarification.  ;-)

Signature

Daniel Morgan
http://www.outreach.washington.edu/ext/certificates/oad/oad_crs.asp
http://www.outreach.washington.edu/ext/certificates/aoa/aoa_crs.asp
damorgan@x.washington.edu
(replace 'x' with a 'u' to reply)

Mark D Powell - 30 Nov 2003 19:05 GMT
> > You can disagree, but I think your are failing to consider how much
> > real world work is done in batch type processing.
[quoted text clipped - 9 lines]
> But your point is valid in a datawarehouse situation. Can we at least
> agree that commits in OLTP loops is almost undoubtedly without merit.

I think we still disagree.  In a data warehouse I would expect there
to be few updaters, but in a large heavily used OLTP where concurrent
access to the same rows is a reality then commits within a loop will
be a necessity for some processes.

There are applications where this may not be necessary and low work
loads allow less frequent commits, but I have yet to look at an MRP or
ERP system that did not contain commits within a loop.

If you this is bad design then I will just refer you to Material
Requirement Planning: The New Way of Life in Production and Inventory
Management by Joseph Orlicky published by McGraw-Hill in 1975.  When I
sought certification by the American Production and Inventory Society,
APICS, this was one of the recommended texts.  It describes what MRP
systems have to do.  Seriously, if you can think of how to perform the
work without using loops, and allowing concurrent update access
without commits, then you can probably make a small fortune since I
know a lot of firms that would be interested.

I think how someone views this argument might be colored by the types
of applications they have experience with.  I have a lot of experience
with applications with high concurrent user work loads that target a
few, large objects and concurrency is a priority.  Smaller systems
would have different characteristics.

The original point was that blanket statements are dangerous because
any one rule may not be suitable for different applications and under
varying work loads.  While I agree there are definite drawbacks to
committing after updating on a per row basis there are situations
where doing so is a necessity and guidelines produced by the DBA for
developers need to allow/account for these situations.  Rather than
say do this or do not do that the guidelines should lay out the side
effects of design decisions based on how Oracle handles the work.
Then it is up to the developers who should know the data better than
the DBA to make the right choices.

IMHO -- Mark D Powell --
Daniel Morgan - 30 Nov 2003 21:34 GMT
>>>You can disagree, but I think your are failing to consider how much
>>>real world work is done in batch type processing.
[quoted text clipped - 11 lines]
>
> I think we still disagree.  

I think we don't.

> The original point was that blanket statements are dangerous

Which is why we don't. Because on this I agree as I've stated.

because
> any one rule may not be suitable for different applications and under
> varying work loads.

And we agree again.

  While I agree there are definite drawbacks to
> committing after updating on a per row basis there are situations
> where doing so is a necessity and guidelines produced by the DBA for
> developers need to allow/account for these situations.

And yet again.

> IMHO -- Mark D Powell --

My point, to repeat myself, was that one should try without commits
inside of loops as an initial approach and only commit inside of a loop
if as a back-up strategy. Too many times commits within a loop are part
of the initial build.

Can we agree to agree?   ;-)

Signature

Daniel Morgan
http://www.outreach.washington.edu/ext/certificates/oad/oad_crs.asp
http://www.outreach.washington.edu/ext/certificates/aoa/aoa_crs.asp
damorgan@x.washington.edu
(replace 'x' with a 'u' to reply)

Joel Garry - 01 Dec 2003 23:47 GMT
> >>>You can disagree, but I think your are failing to consider how much
> >>>real world work is done in batch type processing.
[quoted text clipped - 39 lines]
>
> Can we agree to agree?   ;-)

I'm not sure you've agreed with Mark's statement, which statement
implies that you should not try it first because you must take into
account the overall functioning of the system, which is often less
pure OLTP than one would assume from your repetition.

I've seen it over and over again, especially with Mark's examples of
MRP and ERP systems, they just aren't pure.  Worse, the data warehouse
project that could perhaps partly purify them drags on unfinished
until they get news that, for example, now we can have multiple block
sizes, so why _not_ have everything in one giant instance for
up-to-the-second data?  :-O

Time for an Oracle Purity test!  :-)

jg
--
@home.com is bogus.
Death of Moore's Law, news at 11
http://www.msnbc.com/news/999894.asp?cp1=1
Jeff - 01 Dec 2003 13:47 GMT
>developers need to allow/account for these situations.  Rather than
>say do this or do not do that the guidelines should lay out the side
[quoted text clipped - 3 lines]
>
>IMHO -- Mark D Powell --

Bravo.  Don't treat developers like dumb-a.ses unless you want them to be
dumb-a.ses.
Joel Garry - 01 Dec 2003 23:31 GMT
> >developers need to allow/account for these situations.  Rather than
> >say do this or do not do that the guidelines should lay out the side
[quoted text clipped - 6 lines]
> Bravo.  Don't treat developers like dumb-a.ses unless you want them to be
> dumb-a.ses.

So what do you do if they are dumb-a.ses before they've been treated?
:-)

Seriously, some of the worst I've seen are newbies with a chip on
their shoulder.  They come at it with the attitude that the way they
learned it somewhere else is the way it should be done, and attempting
to lay out guidelines as Mark suggests is just flamebait for them.
Billy V's Big Lead Pipe Treatment becomes unavoidable.  The Dilbertian
side effect is management decides that some overly-restrictive rules
should be placed on everyone.

Personally, I'd rather that developers be encouraged to be creative,
with a filtering process that explicates Oracle's issues with
attempted solutions (in a practical sense that means Explain Plan and
code reviews, as Daniel points out).  This tends to be at odds with
classical formal design processes.

Also, I would expect DBA's to be more familiar with global data issues
than developers, in a large environment.

jg
--
@home.com is bogus.
http://www.signonsandiego.com/news/uniontrib/mon/business/news_mz1b1im.html
Jeff - 02 Dec 2003 13:42 GMT
>So what do you do if they are dumb-a.ses before they've been treated?
>:-)

Re-examine your hiring and/or compensation practices.  In other words, avoid
hiring dumb-a.ses in the first place.  ;-)

>Seriously, some of the worst I've seen are newbies with a chip on
>their shoulder.  They come at it with the attitude that the way they

Newbies or wet-behind-the-ears young'uns fresh out of college that just need
to shut up and mature a bit?  :-)

>Personally, I'd rather that developers be encouraged to be creative,
>with a filtering process that explicates Oracle's issues with
>attempted solutions (in a practical sense that means Explain Plan and
>code reviews, as Daniel points out).  This tends to be at odds with
>classical formal design processes.

What're  "classical formal design processes?"  Because I think code reviews
with an experienced DBA that provides developers with insight in how to do
things better should be welcomed... as long as the DBA isn't full of shatt (or
himself) and/or an a.s about it.  I think experienced developers would, at
least.
Joel Garry - 03 Dec 2003 18:42 GMT
> >So what do you do if they are dumb-a.ses before they've been treated?
> >:-)
>
> Re-examine your hiring and/or compensation practices.  In other words, avoid
> hiring dumb-a.ses in the first place.  ;-)

Guess you've never been a contractor in a gummint shop ;-)

> >Seriously, some of the worst I've seen are newbies with a chip on
> >their shoulder.  They come at it with the attitude that the way they
>
> Newbies or wet-behind-the-ears young'uns fresh out of college that just need
> to shut up and mature a bit?  :-)

Both.  Some of the worst have been customers hiring developers out of
vendors.  So they are new to the shop, have the ability to walk on
water as far as the customer is concerned, and have a lot of
experience at the vendor - which would be one of the same vendors that
do some of the worst deadly sins, often in the name of database
heterogeneity.  And since they can walk on water, they can poke data
and code into the production db... right?

> >Personally, I'd rather that developers be encouraged to be creative,
> >with a filtering process that explicates Oracle's issues with
[quoted text clipped - 7 lines]
> himself) and/or an a.s about it.  I think experienced developers would, at
> least.

By that I mean doing all design work and pseuodocode (or whatever the
particular process favors) up front, and the developers just grind
code.  The diametric opposite is Rapid Application Development (RAD,
or similar buzzwords like Extreme Programming).  Personally I prefer a
middle ground with a bias toward RAD, with care given to watch for the
downsides of each.  To me, the power to have excellent development
lies in the feedback of an iterative cycle.  The top-down approach
coupled with deadlines tends to give code reviews short shrift.  Total
RAD can wind up with an unmaintainable mess, especially when customers
can too easily modify vendor code.

Not that I'm complaining about any of this, it means full employment
for me.  I'm just repelled and fascinated by the horror stories.

jg
--
@home.com is bogus.
http://www.signonsandiego.com/news/uniontrib/tue/business/news_1b2peregrine.html
Mark D Powell - 03 Dec 2003 19:45 GMT
> >So what do you do if they are dumb-a.ses before they've been treated?
> >:-)
[quoted text clipped - 19 lines]
> himself) and/or an a.s about it.  I think experienced developers would, at
> least.

The problem with code walk-throughs is that by the time they happen it
is offen too late to change the design.  Developers need to involve
the DBA with a design walk-thru before code is written.  In my
experience the majority of the time the DBA does not see the code
until there is a production performance problem.  By then major design
changes and different approaches are "too late".

Just another challenge -- Mark D Powell --
Jeff - 04 Dec 2003 14:01 GMT
>The problem with code walk-throughs is that by the time they happen it
>is offen too late to change the design.  Developers need to involve
>the DBA with a design walk-thru before code is written.  In my
>experience the majority of the time the DBA does not see the code
>until there is a production performance problem.  By then major design
>changes and different approaches are "too late".

I agree 100% that a DBA should be part of the design team along with
developers.  If they aren't... if DBA's are only brought in at production time
or after the "design concrete" has set, I'd say that's a management problem,
not a developer problem.  Maybe I'm naive or we've different ideas of what a
"developer" does.

Aside to Joel: I've never worked in a top-down environment... nor have I a lot
of experience (none positive, that's for sure) with working with outside
vendors/consultants.  So, I'm rather speaking to permanent shops with
permanent employees.
Daniel Morgan - 04 Dec 2003 17:39 GMT
Comments in-line.

>>The problem with code walk-throughs is that by the time they happen it
>>is offen too late to change the design.  Developers need to involve
[quoted text clipped - 8 lines]
> not a developer problem.  Maybe I'm naive or we've different ideas of what a
> "developer" does.

You are naive. There are many organizations, I think most, where the DBA
is the last one to see the application.

> Aside to Joel: I've never worked in a top-down environment... nor have I a lot
> of experience (none positive, that's for sure) with working with outside
> vendors/consultants.  So, I'm rather speaking to permanent shops with
> permanent employees.

The one thing that needs to be added to this discussion is the huge
number of Oracle DBAs that think their job consists of install, patch,
lock down, backup and recover. And most pray they never have to do the
recover step because they've never done it before.

If DBAs want to be considered valuable to a development team they must:

1. Learn to read and write PL/SQL well enough to give advice.
2. Develop OO, Java, UML, and other skills to match development.
3. Keep their skills up to the current version.
4. Not treat developers like they are the enemy. There is not harm in
giving developes select on v_$session or access to dbms_profiler.

Too often the DBA's attitude and skills (or the lack thereof) are the
root reason the DBA is excluded.

Signature

Daniel Morgan
http://www.outreach.washington.edu/ext/certificates/oad/oad_crs.asp
http://www.outreach.washington.edu/ext/certificates/aoa/aoa_crs.asp
damorgan@x.washington.edu
(replace 'x' with a 'u' to reply)

Galen Boyer - 05 Dec 2003 04:18 GMT
> Too often the DBA's attitude and skills (or the lack thereof)
> are the root reason the DBA is excluded.

Daniel,

I completely agree with you on this one.  Most DBA departments
(not the dba's themselves) are run as though the DBA departments
are "blessing" the different groups with their godlike knowledge.
Then, when I talk to the DBAs directly, we are all the same.  We
love the technology and want to make great applications and we
are generally just completely interested in ALL phases of the
project.  To me, a DBA and a developer are both the same type of
person just on different skill tracks.

Signature

Galen Boyer

Daniel Morgan - 05 Dec 2003 07:28 GMT
>>Too often the DBA's attitude and skills (or the lack thereof)
>>are the root reason the DBA is excluded.
[quoted text clipped - 9 lines]
> project.  To me, a DBA and a developer are both the same type of
> person just on different skill tracks.

You are correct and you point out another problem. The division of DBAs
and developers into different departments with different budgets and
different managers. Can't think of a better way to NOT create a team
atmosphere.

Remember my 4th point ... managers that can't or won't? This screams of
the same issue. What person in upper management thinks it is a good idea
to separate DBAs from developers? And why isn't that person the one
being downsized out of his or her job? I mean any reason other than the
issue of who plays golf with who?

Signature

Daniel Morgan
http://www.outreach.washington.edu/ext/certificates/oad/oad_crs.asp
http://www.outreach.washington.edu/ext/certificates/aoa/aoa_crs.asp
damorgan@x.washington.edu
(replace 'x' with a 'u' to reply)

Galen Boyer - 05 Dec 2003 15:57 GMT
> The division of DBAs and developers into different departments
> with different budgets and different managers. Can't think of a
> better way to NOT create a team atmosphere.

We have this at my place of work.  The wall between my database
team and the DBA team of our parent company was such that I
wasn't given a lick of access to any of our development database
machines.  I asked for dba access to the development machines
they wanted us to use and they got all up in arms about security
and other production concerns.  These machines were to be used by
my team and my team only.  They have never once asked my about my
designs or layouts or anything.  The only thing they wanted to
make sure of was that we were going through their layers to get
anything done.  Here is my answer in a thread that I was involved
in.  The thread I was involved in is probably reconstructed by
many here from similar pleas for sane viewpoints from the DBA
crews.

   To your question about dba access, I will answer from my
   personal viewpoint. (ie, I don't want to talk for our overall
   architect) . I don't understand how the database architect
   for the whole project isn't given full access to all
   databases designated for development purposes for the
   project.  These are development instances.  They shouldn't be
   locked down as production instances.  The only thing that
   should be locked down is the way the app is interacting with
   the database, because we are building a production app.  We
   need to be certain that we hamper the application in exactly
   the same way it will be hampered in production.  But, we
   don't need to hamper the development process with the same
   restrictions.  When a construction crew is building a new
   home, they don't put the locks on the doors and windows and
   make the construction crew knock on the door to get in to do
   their work, but they certainly make sure the home is put
   together, just like it will be when it is sold.  Before it is
   put on the market and sold, the security against even the
   guys that built it is then, and only then, put in place.

Signature

Galen Boyer

Daniel Morgan - 05 Dec 2003 19:03 GMT
>>The division of DBAs and developers into different departments
>>with different budgets and different managers. Can't think of a
[quoted text clipped - 33 lines]
>     put on the market and sold, the security against even the
>     guys that built it is then, and only then, put in place.

So somehow the DBA team expects the developers to develop a good
application without access to DBMS_PROFILER, without access to v$ magic
views, and without access to other tables and views critical to
understanding what is happening and optimizing performance and
scalability. Good one.

Then I imagine they sit around drinking coffee and whining about the
poor quality of apps developed by your team.

Once again the issue goes straight to management incompetence. The
managers in charge of these teams are the root source of the problem.
They are doing nothing to foster cooperation.

Signature

Daniel Morgan
http://www.outreach.washington.edu/ext/certificates/oad/oad_crs.asp
http://www.outreach.washington.edu/ext/certificates/aoa/aoa_crs.asp
damorgan@x.washington.edu
(replace 'x' with a 'u' to reply)

Galen Boyer - 05 Dec 2003 19:30 GMT
> So somehow the DBA team expects the developers to develop a
> good application without access to DBMS_PROFILER, without
[quoted text clipped - 4 lines]
> Then I imagine they sit around drinking coffee and whining
> about the poor quality of apps developed by your team.

Actually, that is about it. After many battles and executives
learning how much my team was hampered in trying to get our app
tested on our different deployment environments, I now have full
control of now only the development instances, but also complete
access to the production instances.  (I still relegate the
production DBA responsibilities to the DBA team)

The business folks paying the checks weren't happy to learn that
bureucracy was in the way of progress.  They told the dba team to
just open up the doors, completely.

> Once again the issue goes straight to management
> incompetence. The managers in charge of these teams are the
> root source of the problem. They are doing nothing to foster
> cooperation.

Its really that my parent company has never had a group willing
to fight for what was needed.  Everybody was always located on
sight in the motherland headquarters downtown New York.  The
subsidiary I work for is located in Boston.  The people making
these ridiculous decrees had to answer to a group of developers
and architects that they didn't manage.  When the questions
started coming out and climbing up the email escalation threads,
the business folks started waking up to the power that the
internal IT group held, and they didn't like it, one bit.

Signature

Galen Boyer

JEDIDIAH - 06 Dec 2003 01:24 GMT
>> So somehow the DBA team expects the developers to develop a
>> good application without access to DBMS_PROFILER, without
[quoted text clipped - 15 lines]
> bureucracy was in the way of progress.  They told the dba team to
> just open up the doors, completely.

I can't imagine the production DBA's were any too happy about this.
The last shop I worked in that was subjected to such shenanigans
found it's entire Oracle DBA and Unix Sysadmin staff flee in a mass
exodus.

NO ONE wanted to held responsible for possible phantom screwups
coming from the development side of the shop.

[deletia]

You traded one insanity for another.

Signature

There's no reason not to contribute to the stone soup if those    |||
contributions are not critical to your competitive edge.         / | \

Galen Boyer - 08 Dec 2003 03:39 GMT
>> The business folks paying the checks weren't happy to learn
>> that bureucracy was in the way of progress.  They told the dba
[quoted text clipped - 7 lines]
> NO ONE wanted to held responsible for possible phantom screwups
> coming from the development side of the shop.

Well, if the DBA team were even remotely interested in being part
of our development solution, if they wanted to do code reviews
and analyze database designs and basically get involved in our
development effort, then, well I'd have been all ears.  I invited
them into our CVS code base numerous times and they declined.
Our code has been in source control since day one.  Not one, let
me count them for you, not one DBA looked at it.  I installed an
internal newserver where we discussed all our technical issues at
a level you see here.  I invited them into that arena, they
declined. I offered to work on a database module in CVS with them
so we could have a code base that all those doing database work
in our company could use.  They declined. I never, ever wanted to
not have a dba crew around.  And, you know what?  I still have a
dba crew around.  They are in charge of our production instance.
I have access to it, can do data uploads and patch our instances,
but they take care of backup and recovery.  With the level of
their involvement in the development cycle, which was zilch, this
is exactly what they should do, and this is exactly what they
ONLY should do.

> You traded one insanity for another.

You are making a judgement based on different circumstances or
just based on the titles of the characters involved in the story.

Signature

Galen Boyer

Paul Drake - 06 Dec 2003 04:42 GMT
> > The division of DBAs and developers into different departments
> > with different budgets and different managers. Can't think of a
[quoted text clipped - 33 lines]
>     put on the market and sold, the security against even the
>     guys that built it is then, and only then, put in place.

what privileges do you actually require?

What if the DBA (person) removed all of the privileges from the DBA
role except for create session, table, type, view, trigger, sequence,
synonym and alter session?

the problem with granting an app_owner schema the role DBA is that
then the application is coded depending upon the DBA role (and
usually, all of the sys_privs that are in that role, that the account
invariably grants itself directly). it is such a PITA to get changes
made to remove queries that hit the dba_ views (such as
dba_cons_columns for RI errors). If the developers can't code against
the dba_% views, but are limited to the all_ views, you don't have as
many issues when the code runs on a qa db where the app owner account
does not have the DBA role granted to it.

If you are using a single app_owner schema, you don't need any of the
ANY privs.
Yes, you need dictionary access in development, so the developer role
gets dictionary access.

You only think that you need the DBA role granted to you.
You most likely do not.
The more you read about security, the more that you'll agree with this
viewpoint (IMHO).

This is a gross oversimplification, I'm sure that you can find
exceptions, but I believe it to be generally true.

Give the developers enough privileges to create objects in the own
schemas in dev to get their work done, but not enough sys_privs to
alter the app_owner schema(s)  themselves.

Pd
Daniel Morgan - 06 Dec 2003 08:05 GMT
Comments in-line.

> what privileges do you actually require?

Select on v$ magic views required to see what is actually happening. One
can start with v_$reserve_words and v_$open_cursor and if you have any
imagination you can come up with dozens more.

> What if the DBA (person) removed all of the privileges from the DBA
> role except for create session, table, type, view, trigger, sequence,
> synonym and alter session?

If you read Oracle instructions, which apparently few do, no one should
be granted the DBA role. The DBA role, like CONNECT and RESOURCE are
intended in the same manner as SCOTT/TIGER. They are supposed to be
dropped following installation and new roles created appropriate to
employee job requirements. The rule is give employees the privileges
they need to do their job. No privilege they don't need any more than
denying privileges they do need.

> the problem with granting an app_owner schema the role DBA is that

This is a red herring. The DBA role, as provided by Oracle is supposed
to be dropped.

> The more you read about security, the more that you'll agree with this
> viewpoint (IMHO).

In development. Sorry I don't buy you argument. In test yes. In staging
yes. In production absolutely. But if developers do stupid things in
development their work fails in test. So what? And that is what code
reviews are supposed to catch long before anything goes to test. What
you appear to be doing is handcuffing everyone because someone might do
something if you let them.

> This is a gross oversimplification, I'm sure that you can find
> exceptions, but I believe it to be generally true.

I agree that it is a gross oversimplification. But I do not agree that
it is generally true. I think a more generally true statement would be
that DBAs inhibit the ability of developers to do their job and claim it
relates to security. Then whine about the poor quality of what
developers develop when it finally gets to production.

> Give the developers enough privileges to create objects in the own
> schemas in dev to get their work done, but not enough sys_privs to
> alter the app_owner schema(s)  themselves.
>
> Pd

Why? Tell me exactly what it is the competently hired, competently
trained, and competently managed developers are going to do if they
actually modify the app_owner schema in development?

The only come up with two possible answers.
1. Produce better applications
2. Not hate DBAs

Both sound like good outcomes to me.
Signature

Daniel Morgan
http://www.outreach.washington.edu/ext/certificates/oad/oad_crs.asp
http://www.outreach.washington.edu/ext/certificates/aoa/aoa_crs.asp
damorgan@x.washington.edu
(replace 'x' with a 'u' to reply)

Galen Boyer - 06 Dec 2003 15:12 GMT
> the problem with granting an app_owner schema the role DBA is
> that then the application is coded depending upon the DBA role
[quoted text clipped - 6 lines]
> runs on a qa db where the app owner account does not have the
> DBA role granted to it.

You are missing my point.  I never want the user that the
application will log in as to have anything but the priviledges
that will be granted to it in public.  What I want is a user that
has dba priviledges (or a form thereof) that can be used by me
and the development crew for the sole purpose of modifying the
database for developing the app.  I, most definitely, want to
hamper the application schema exactly as I plan to in production.

Signature

Galen Boyer

Daniel Morgan - 06 Dec 2003 15:59 GMT
>>the problem with granting an app_owner schema the role DBA is
>>that then the application is coded depending upon the DBA role
[quoted text clipped - 14 lines]
> database for developing the app.  I, most definitely, want to
> hamper the application schema exactly as I plan to in production.

What Paul Drake and so many others miss is that the security policy of a
company should be a written document. And the security rights and
priviileges of an application should be part of a written specification.

If developers are required to create or modify applications based on
those two documents then no application they create or modify can
possibly violate the agreed security policies and specifications.

The only thing the developers could possibly do is make their jobs a bit
easier. Little things like looking up reserved words, checking to make
sure that all of their dynamic SQL is using bind variables, looking at
the numbers of hard and soft parses: In short all of the things Tom Kyte
does on his asktom website to demonstrate that things are working as
they should.

The sad fact is, and since I don't know you Paul I'm not pointing at you
so don't take this personally, the DBAs that are most paranoid about
giving developers privileges in development environments, are the DBAs
that know the least about development: The ones that can't actually
write and debug PL/SQL.

Signature

Daniel Morgan
http://www.outreach.washington.edu/ext/certificates/oad/oad_crs.asp
http://www.outreach.washington.edu/ext/certificates/aoa/aoa_crs.asp
damorgan@x.washington.edu
(replace 'x' with a 'u' to reply)

JEDIDIAH - 07 Dec 2003 04:59 GMT
[deletia]
> The sad fact is, and since I don't know you Paul I'm not pointing at you
> so don't take this personally, the DBAs that are most paranoid about
> giving developers privileges in development environments, are the DBAs
> that know the least about development: The ones that can't actually
> write and debug PL/SQL.

No, it is far more likely that the paranoid DBA has little or no
respect for the developers. Developer quality varies widely.
Some can even be entrusted with the DBA role on production systems.
However, most can't even be trusted to wipe their own noses.

It is safer to be paranoid.

"can't actually write and debug PL/SQL" is a common problem all round.

Signature

There's no reason not to contribute to the stone soup if those    |||
contributions are not critical to your competitive edge.         / | \

Daniel Morgan - 07 Dec 2003 07:02 GMT
> [deletia]
>
[quoted text clipped - 12 lines]
>
> "can't actually write and debug PL/SQL" is a common problem all round.

You've posted, Paul's posted, others have posted.

I have 10+ years working as an Oracle DBA and I am still looking for
concrete, real-world, examples of problems caused in production by
developers that were allowed to manage their own sandbox.

If there is something here other than paranoia ... name names. What
actually happened? And if a properly written and approved design with
code reviews would have solved the problem you don't have a point other
than a lack of decent procedures.

I await those examples. I suspect many developers watching here are too.

Signature

Daniel Morgan
http://www.outreach.washington.edu/ext/certificates/oad/oad_crs.asp
http://www.outreach.washington.edu/ext/certificates/aoa/aoa_crs.asp
damorgan@x.washington.edu
(replace 'x' with a 'u' to reply)

Galen Boyer - 08 Dec 2003 03:40 GMT
> No, it is far more likely that the paranoid DBA has little or
> no respect for the developers.

And somehow DBAs must have respect from developers?  Why?
Because they have the title of DBA?

> Developer quality varies widely.

DBA quality varies widely.

Signature

Galen Boyer

Paul - 16 Dec 2003 00:32 GMT
galenboyer@hotpop.com says...

> > No, it is far more likely that the paranoid DBA has little or
> > no respect for the developers.

> And somehow DBAs must have respect from developers?  Why?
> Because they have the title of DBA?

Some of them appear to think so!


> > Developer quality varies widely.

> DBA quality varies widely.

Amen to that - as a developer who wants to make the switch to DBA, I
can't think of anyone better than an ex-developer to know the faults
that DBA's seem to develop - maybe I'll be cursing developers in the
future, but AFAICS, what Daniel Morgan is saying is dead on - you give
the developers the rope that they can use to hang themselves *_ON
DEVELOPMENT SYSTEMS_*. That's the whole point of being a developer.

I also find the idea of a hermetic seal between developers and DBA's who
are supposed to be working for the same company on the same product a
bit difficult - maybe I'm better suited to smaller shops where I can be
both simultaneously, or at least where there is an overlap of roles, and
one side understands (or at least tries to) the other!

Paul...


Signature

plinehan  x__AT__x  yahoo  x__DOT__x  com

C++ Builder 5 SP1, Interbase 6.0.1.6 IBX 5.04 W2K Pro

Please do not top-post.

Niall Litchfield - 16 Dec 2003 08:56 GMT
> I also find the idea of a hermetic seal between developers and DBA's who
> are supposed to be working for the same company on the same product a
> bit difficult - maybe I'm better suited to smaller shops where I can be
> both simultaneously, or at least where there is an overlap of roles, and
> one side understands (or at least tries to) the other!

I think you are exactly correct here, though DBAs and Developers do tend to
have somewhat different mindsets. I speculate that this comes from the
pressure on developers to deliver 'solutions' extremely rapidly and to tight
budgets whilst preferably using new technologies so as not to be obsolete.
DBAs tend to be under pressure to provide available, secure, highly
performing data stores.

Signature

Niall Litchfield
Oracle DBA
Audit Commission UK

Paul Drake - 06 Dec 2003 22:43 GMT
> > the problem with granting an app_owner schema the role DBA is
> > that then the application is coded depending upon the DBA role
[quoted text clipped - 14 lines]
> database for developing the app.  I, most definitely, want to
> hamper the application schema exactly as I plan to in production.

and for that, you do not require the sys_privs that are granted by
default to the dba role created by default in a new database.

you do not need the "ANY" sys_privs to do what you are describing
above.
if you have the create sys_privs granted directly or via a role, your
can create anything that you can dream of under that app_owner schema.

its only when you reference objects outside of that schema that the
"ANY" sys_privs are thought to be required.

Pd
Paul Drake - 07 Dec 2003 05:42 GMT
> > the problem with granting an app_owner schema the role DBA is
> > that then the application is coded depending upon the DBA role
[quoted text clipped - 14 lines]
> database for developing the app.  I, most definitely, want to
> hamper the application schema exactly as I plan to in production.

oh.
please allow me to take a step back for a second.

we now have 3 user types, so let us assume that roles are in use for
simplicity.

you have an app_owner schema role.
you have an application user role (unprivileged).
you have a dba role (although not necessarily the role named "DBA"
that is pre-built by dbca).

you want an account that has some sort of dba role that can alter
database properties, and have access to the init.ora or spfile.

your existing dba is unavailable, overworked, not competent,
uninterested or just plain has no clue and nothing to offer in terms
of background, experience, etc.

I can see you wanting to be able to make changes to the database
config so as to help optimize how the app code runs. you want the
project to succeed.

I could also see where an existing dba (group) might not want you
tweaking parameters without justification, this sounds not so much
like a communal development environment, but a play area for you to
hone your dba skills. personal workstation and laptops are a great
place to experiment. If you have to support a horde of developers with
this dev db, it might not be the best place for you to develop/hone
your dba skills.

one possible approach is to get say 10% of your time allocated to the
dba group, and gain their respect, work with them to optimize your
development environment.

another possible approach is to completely break from having any
involvement from  your dbas employed by your company. you will likely
lose out on assistance that they could provide.

most likely, you do not want a development crew making database
changes en masse.
if anything, give them alter session so that they can muck with things
as far as their session is concerned, but not muck with other user's
sessions, or instance-wide settings.

I'd prefer if you not use the term "public" but substitute the term
"production".
to me, "PUBLIC" is something that is granted (improperly?) where
security is of absolutely no concern - dual, perhaps - where anyone
that has a session, has read/execute on the object - or worse (think
dba_tab_privs).

I am honestly trying to be constructive here, and might have as much
to gain from  these thought experiments as you.

Pd
Daniel Morgan - 07 Dec 2003 07:15 GMT
Comments in-line.

> you want an account that has some sort of dba role that can alter
> database properties, and have access to the init.ora or spfile.
[quoted text clipped - 14 lines]
> this dev db, it might not be the best place for you to develop/hone
> your dba skills.

If I may jump in for Galen here, he's probably smarter than me and in
bed ... what you've written is not the point at all. But your attitude
is exactly what causes developers so much grief.

Developers don't have machines at home that are like those at work. I
don't know a single developer who has a personal HP9000 (well I have two
but that's not the point a do mostly DBA work and teaching). Nor are
there many with Sun 450s or machines running AIX or OS/390, etc. So lets
not pretend one can learn on their personal PC that which will best
optimize an application in a business environment.

> one possible approach is to get say 10% of your time allocated to the
> dba group, and gain their respect, work with them to optimize your
> development environment.

Another possible approach would be for one of you that believes in
locking down a development environment to put forth actual, factual
examples of things that have gone wrong. And I don't mean things that
potentially could or might go wrong. And I don't mean urban myths you
heard about at a Christmas party. I mean things that actually have
happened to you.

Much like the detectives on Dragnet ... I want the facts.   ;-)

> another possible approach is to completely break from having any
> involvement from  your dbas employed by your company. you will likely
> lose out on assistance that they could provide.

Funny cooperation isn't one of your possible approaches. Why? My
suggested approach, and the one I use is to ask for justification and
grant anything and everything that will not cause a problem when the
code movest to test. If they mess it up ... that is their right.
Interestingly though ... it has been more than three years since I've
seen a development box blow up because of what developers did. And the
last one that did was when a junior member of the DBA team tried to show
off at a level above her abilities.

> most likely, you do not want a development crew making database
> changes en masse.

And if they have management why would they?

> if anything, give them alter session so that they can muck with things
> as far as their session is concerned, but not muck with other user's
> sessions, or instance-wide settings.

Why? Provide one actual example of something that actually happened and
how this caused a problem.

> I am honestly trying to be constructive here, and might have as much
> to gain from  these thought experiments as you.
>
> Pd

I'm sure you are. But the divide between you and me (and I am mostly a
DBA) is as wide as the Grand Canyon. Between you and most deveopers the
size of the Pacific Ocean.

I understand your concern about people doing stupid things. I have the
same concerns. But I am still asking for and awaiting actual examples of
things that have gone wrong. From where I sit you have crossed the line
from concern into paranoia.

And not once has anyone addressed my statements about granting access to
developers to the v$ magic views and many of the DBMS and UTL packages
and I would bet a Latte' at Starbucks you don't give your developers
access to them either.

Signature

Daniel Morgan
http://www.outreach.washington.edu/ext/certificates/oad/oad_crs.asp
http://www.outreach.washington.edu/ext/certificates/aoa/aoa_crs.asp
damorgan@x.washington.edu
(replace 'x' with a 'u' to reply)

Billy Verreynne - 08 Dec 2003 05:44 GMT
Daniel Morgan <damorgan@x.washington.edu> wrote

> And not once has anyone addressed my statements about granting access to
> developers to the v$ magic views and many of the DBMS and UTL packages
> and I would bet a Latte' at Starbucks you don't give your developers
> access to them either.

Well, on our DEV platform I have given the developer accounts select
access to the v$ magic views in a vain hope that with a little bit of
self empowerment, they will get to know Oracle better and  even use it
better.

But then I still get stuff like this in production:
begin
 insert statement (trigger updates the seq)

 select max(seq) into :s from table;
 
 return( :s )
end;

Code review? Not here. Every developer writes perfect code. Why the
hell get the DBA involved at all (except when you want to use him as a
friggen reference manual when too lazy to look up the syntax for an
Oracle function or SQL statement).

--
Billy
Jeff - 08 Dec 2003 13:47 GMT
>Daniel Morgan <damorgan@x.washington.edu> wrote
>
[quoted text clipped - 16 lines]
>  return( :s )
>end;

This isn't evidence of developers that aren't Oracle literate.  This is
evidence of under-competent (sic) developers.  Code reviews can be mandated by
management--and if a DBA brought this kind of code to a competent IT manager,
he/she should certainly see the need for them.
Daniel Morgan - 08 Dec 2003 17:24 GMT
> Daniel Morgan <damorgan@x.washington.edu> wrote
>
[quoted text clipped - 24 lines]
> --
> Billy

That's half a latte'. How about DBMS_PROFILER and other built-in packages?

If you get it right send me your postal address off-line and the latte'
is yours.

Signature

Daniel Morgan
http://www.outreach.washington.edu/ext/certificates/oad/oad_crs.asp
http://www.outreach.washington.edu/ext/certificates/aoa/aoa_crs.asp
damorgan@x.washington.edu
(replace 'x' with a 'u' to reply)

Billy Verreynne - 09 Dec 2003 05:42 GMT
Daniel Morgan <damorgan@x.washington.edu> wrote

> That's half a latte'. How about DBMS_PROFILER and other built-in packages?

They don't know how to use expain plan. I have tried numerous times to
get them to use TKPROF. Access to DBMS_PROFILER? They won't know what
to do with it.

FWIW, they do have access to workspaces, replication, directory
objects and a bunch of other stuff that I as an Oracle developer would
love to have my hands on. Also trying to get them to use OEM and
JDeveloper. But the only thing they can use is TOAD.... <sigh>

> If you get it right send me your postal address off-line and the latte'
> is yours.

Does it come with several cases of beer? ;-)

--
Billy
Daniel Morgan - 09 Dec 2003 07:09 GMT
> Daniel Morgan <damorgan@x.washington.edu> wrote
>
>>That's half a latte'. How about DBMS_PROFILER and other built-in packages?
>
> They don't know how to use expain plan.

Shame shame shame!

Better than what I'm really thinking.

 I have tried numerous times to
> get them to use TKPROF. Access to DBMS_PROFILER? They won't know what
> to do with it.

I have a rule ... no code goes into production without accompanying
explain plans and DBMS_PROFILER runs. It is part of the rules of
engagement and should be mandatory everywhere.

Any developer can learn Explain Plan in five minutes: DBMS_PROFILER in
thirty or less.

>>If you get it right send me your postal address off-line and the latte'
>>is yours.
[quoted text clipped - 3 lines]
> --
> Billy

I keep Chimay in the frige but it doesn't ship well.

Signature

Daniel Morgan
http://www.outreach.washington.edu/ext/certificates/oad/oad_crs.asp
http://www.outreach.washington.edu/ext/certificates/aoa/aoa_crs.asp
damorgan@x.washington.edu
(replace 'x' with a 'u' to reply)

Paul Drake - 09 Dec 2003 16:39 GMT
> > Daniel Morgan <damorgan@x.washington.edu> wrote
> >
[quoted text clipped - 26 lines]
>
> I keep Chimay in the frige but it doesn't ship well.

That, is a difficult task, as it keeps disappearing.
East India Pale Ales ship very well, like from the UK, to Mumbai,
across India.

I checked out your article on DBMS_PROFILER last night.
NYOUG has an excellent technical journal, and is always looking for
authors to submit papers/articles (http://www.nyoug.org) if you're
interested.

btw - the link to the Oracle Trace Analyzer utility is obsolete.
there is a download link in the Metalink note 224270.1
http://metalink.oracle.com

hth.

Pd
JEDIDIAH - 11 Dec 2003 16:03 GMT
> Comments in-line.
>
[quoted text clipped - 27 lines]
> not pretend one can learn on their personal PC that which will best
> optimize an application in a business environment.

What makes you think that a developer will necessarily have reasonable
development hardware even at the office? Any place that I have ever
worked (or ever supported) has had a shortage of suitable development
machines.

Quite often, "someone's desktop" actually ends up being a dev or QA
server.

I've seen this in environments where the corp was already spending
millions on Oracle licenses and server hardware.

[deletia]

Signature

There's no reason not to contribute to the stone soup if those    |||
contributions are not critical to your competitive edge.         / | \

Galen Boyer - 08 Dec 2003 03:10 GMT
> I am honestly trying to be constructive here,

If only I believed you.
Signature

Galen Boyer

Joel Garry - 05 Dec 2003 19:27 GMT
> Comments in-line.
>
[quoted text clipped - 23 lines]
> lock down, backup and recover. And most pray they never have to do the
> recover step because they've never done it before.

Don't forget "tune system for high hit ratios and spend weekends
making sure each table fits in one big initial extent."  :-)


> If DBAs want to be considered valuable to a development team they must:
>
[quoted text clipped - 6 lines]
> Too often the DBA's attitude and skills (or the lack thereof) are the
> root reason the DBA is excluded.

You are quite right, but the downside is spreading the skill set too
thin.  For many environments, I think there should be quite different
production and development DBA's, the latter being more directly
skilled in your list, perhaps even being titled "senior programmer."
Hmmm, that came out different from what I started out to post...

jg
--
@home.com is bogus.
"Someone once said 'Less is More.'  That person was an idiot.  You
don't ask your boss for less money or less vacation time." - radio ad.
Joel Garry - 05 Dec 2003 01:22 GMT
> >The problem with code walk-throughs is that by the time they happen it
> >is offen too late to change the design.  Developers need to involve
[quoted text clipped - 13 lines]
> vendors/consultants.  So, I'm rather speaking to permanent shops with
> permanent employees.

The particular environment I was referring to was a very large
permanent shop with permanent employees.  They had bought an off the
shelf package and had a large effort to modify it to their purposes,
both with contractors and permanent employees.  In fact, there were
two large shops, in competition to do the same thing.  One shop was
using a layered approach, putting all new objects in separate schemata
with private synonyms, while the other hacked away.  They both were
using Oracle tools to reverse-engineer the product, and both were
traditional top-down shops, newly coming to Oracle from COBOL, and
neither had the type of dba that would be necessary for what we are
discussing here.  Eventually one shops methodology took over, due to
political reasons.  Somehow, it all got done, albeit slowly.  Which is
a lot more than can be said for their DW project.

"Management problem" is an understatement.

jg
--
@home.com is bogus.
http://www.sumeiyu.com/
Jeff - 05 Dec 2003 21:52 GMT
>The particular environment I was referring to was a very large
>permanent shop with permanent employees.  They had bought an off the
[quoted text clipped - 11 lines]
>
>"Management problem" is an understatement.

You are describing my personal nightmare... one that I've fought tooth and
nail against from the bottom up.  I believe we could design and develop a
brand new system that'd be as good or better than what you'd end up with by
buying and extensively modifying something.  My personal bias.
Daniel Morgan - 05 Dec 2003 22:55 GMT
>>The particular environment I was referring to was a very large
>>permanent shop with permanent employees.  They had bought an off the
[quoted text clipped - 16 lines]
> brand new system that'd be as good or better than what you'd end up with by
> buying and extensively modifying something.  My personal bias.

From my experience you'd be right far more than wrong. I am aware of a
system in King County Washington, based on Informix. They had a quote to
build from scratch for $700,000. They bought an OTS prouct for $70,000.
They have now spent 2.1 million tax payer dollars trying to modify the
beast.

Oh yeah. And no one likes it either.
Signature

Daniel Morgan
http://www.outreach.washington.edu/ext/certificates/oad/oad_crs.asp
http://www.outreach.washington.edu/ext/certificates/aoa/aoa_crs.asp
damorgan@x.washington.edu
(replace 'x' with a 'u' to reply)

Paul - 16 Dec 2003 00:40 GMT
damorgan@x.washington.edu says...

>  From my experience you'd be right far more than wrong. I am aware of a
> system in King County Washington, based on Informix. They had a quote to
> build from scratch for $700,000. They bought an OTS prouct for $70,000.
> They have now spent 2.1 million tax payer dollars trying to modify the
> beast.

Where would we get work but for foo-bars like that?

Paul...

Signature

plinehan  x__AT__x  yahoo  x__DOT__x  com

C++ Builder 5 SP1, Interbase 6.0.1.6 IBX 5.04 W2K Pro

Please do not top-post.

Paul - 16 Dec 2003 00:18 GMT
Mark.Powell@eds.com says...

> Then it is up to the developers who should know the data better than
> the DBA to make the right choices.

Are you suggesting that a DBA shouldn't have a notion about what data is
in his databases, and not know why it is there and for what purpose it
is being used?

OK, maybe you didn't say "not have a notion" but you definitely said
"should know the data better than the DBA".

I disagree - a DBA should know more than application programmers about
what's in the DB's he manages and why - how else can he make
recommendations about how "things" could be done far better on a far far
higher level than, perhaps, the programmers see the data? "Things"
meaning "System organisation", "optimisation", &c. &c.

Paul...


> IMHO -- Mark D Powell --

Signature

plinehan  x__AT__x  yahoo  x__DOT__x  com

C++ Builder 5 SP1, Interbase 6.0.1.6 IBX 5.04 W2K Pro

Please do not top-post.

Galen Boyer - 16 Dec 2003 15:36 GMT
> Mark.Powell@eds.com says...
>
[quoted text clipped - 14 lines]
> programmers see the data? "Things" meaning "System
> organisation", "optimisation", &c. &c.

I'm not sure how a dba is supposed to do this.  If the DBA is
also the data architect, then, maybe.  But then, the DBA will
spend all his time exploring the business instead of architecting
the database.  I think it is a team effort.  The DBA should be
able to depend on a data architect to give him the needed
information so he can make the decisions his application's
datastore needs.

The developers are going to be the experts at what the sqlcode is
doing and the DBA should be able to depend on them so he can help
them right more performant code and also stay away from bad code.

I guess, nobody should think of themselves as the "sole" source
of info.

Signature

Galen Boyer

Niall Litchfield - 16 Dec 2003 16:23 GMT
> > Mark.Powell@eds.com says...
> >
[quoted text clipped - 26 lines]
> doing and the DBA should be able to depend on them so he can help
> them right more performant code and also stay away from bad code.

writing code that peforms acceptably means knowing things like (assume OLTP
for the moment)

How many transactions/day do we do?
When do we do them?
What is a transaction anyway?
How many customers do we have? And products.
How many order items would an order have on average - and what sort of range
of values might items/order take.
What batch processes are run against this data set. How long do they have to
run.
as well as how to write SQL\PL/SQL\JAVA\ASP

now you might call this design and have it as a separate role but my hunch
is that DBAs tend to know these sorts of things far more than developers. Of
course if you actually *sell* your product then you pretty much have no way
of answering these questions, and then folk worry about software quality of
3rd party apps.

> I guess, nobody should think of themselves as the "sole" source
> of info.

I agree entirely.

Signature

Niall Litchfield
Oracle DBA
Audit Commission UK

Joel Garry - 27 Nov 2003 00:44 GMT
> > Hello,
> >
[quoted text clipped - 20 lines]
> activity for a table that has low select activity, would best be
> served by one or very few commits.

The problem here is beyond semantics.  You are likely breaking up
logical transactions into several physical transactions, which means
you must go beyond the inherent Oracle mechanisms to maintain
atomicity.  Some people would be horrified.  As someone who deals with
your example every day, I say you are exactly right, and wonder why
"transaction levels" have never been implemented into the mechanisms
(the stuff [4GL/OCI generator] I work with actually had that
documented as a future enhancement, but the future never seems to have
arrived; there is a language construct for controlling how often a set
of rows will commit, so it is easy to fix per loop/per site without
major code hassles).

With what we have in Oracle now, there is a legitimate trade-off
between your example and "correct" transactional programming.  What
Dusan needs is a rule that codifies that.  There have certainly been
some amazingly bad examples of loopy commit.

> Be careful in the process of trying to get something into the hands of
> developers that you do not over simplify the problem and end up making
> a statement that you later consider "dumb".  Writing for developers is
> not an easy task.

You got that right!  Beyond that, it is difficult to make such a list
without sounding like, shall we say, authors who have been pummelled
on this group for making lists.

Dusan, I encourage you to do this, and volunteer to edit for good
English if you email me (s/home.com/cox.net).  Apologies in advance if
I'm a bit slow reading email.

jg
--
@home.com is bogus.
"The cursor..For loop approach for the update was calculated to take
53.7 years
to complete!"  - Ask Tom posting.
Alkos - 27 Nov 2003 09:23 GMT
> Hello,
>
[quoted text clipped - 13 lines]
>
> Dusan Bolek

Datatypes is an issue I'm constantly dealing with :-((

Using a unproper (is that correct English ?) datatype for column
e.g. NUMBER for a date (number of days/seconds elapsed)
      VARCHAR2 instead of NUMBER for a numeric

Being unaware of nearly deprecated datatypes and keeping on using them
(VARCHAR, RAW)

Oversizing variable length datatypes

hth
Alkos
Sybrand Bakker - 28 Nov 2003 23:11 GMT
>Datatypes is an issue I'm constantly dealing with :-((
>
[quoted text clipped - 4 lines]
>Being unaware of nearly deprecated datatypes and keeping on using them
>(VARCHAR, RAW)

Let's shake hands. I have to deal with an in-house developed app
(yuck) in which all number(10) are stored as varchar2s, left-padded.
Everytime when I set up a query filter in Toad I have to take those
dratted leading 0s into account. I will remain silent about the
performance impact.

--
Sybrand Bakker, Senior Oracle DBA
Alkos - 01 Dec 2003 08:24 GMT
> >Datatypes is an issue I'm constantly dealing with :-((
> >
[quoted text clipped - 13 lines]
> --
> Sybrand Bakker, Senior Oracle DBA

Hi Sybrand,

I think there is a conspiracy/contest between developers to make the use of
datatypes
to most obfuscated possible.

A recent example (simple one but hidden in a sql script of 1000 lines)
Select bla bla
from table1,table2
where join predicates
and table1.status!=1;

=> ORA 1722 (status was VARCHAR2 but now contains only numeric)

Cheers,
Alkos
Alkos - 01 Dec 2003 14:11 GMT
> >Datatypes is an issue I'm constantly dealing with :-((
> >
[quoted text clipped - 13 lines]
> --
> Sybrand Bakker, Senior Oracle DBA

Hi again,

Let me ask you something  . . .
. . . how does this app perform basic calculations (+,-,* or even /) ?

Alkos
Marc Blum - 27 Nov 2003 21:16 GMT
(1)
SELECT DISTINCT ...

(2)
...  WHERE <attribute> IN (SELECT DISTINCT ...)

(3)
LOOP
  EXECUTE IMMEDIATE some_sql;
END LOOP;

(4)
SELECT MAX(id) + 1 AS new_id FROM big_table

(5)
WHERE TRUNC(date_attr) = TRUNC(date_variable)

--
Marc Blum
mailto:blumNOSPAM@marcblum.de
http://www.marcblum.de
Cris Carampa - 28 Nov 2003 08:36 GMT
> (1)
> SELECT DISTINCT ...

Sorry for the naive question: what's wrong with SELECT DISTINCT?

Kind regards,

Signature

Cris Carampa (spamto:cris119@operamail.com)

"Poveri fanatici comunisti, noglobal e affetti dalla sindrome
anti-microsoft" (gli utenti Linux secondo un poster di ICOD)

Alkos - 28 Nov 2003 09:43 GMT
> > (1)
> > SELECT DISTINCT ...
[quoted text clipped - 8 lines]
> "Poveri fanatici comunisti, noglobal e affetti dalla sindrome
> anti-microsoft" (gli utenti Linux secondo un poster di ICOD)

Most of the time, people use SELECT DISTINCT to get unique rows because
they don't want to "bother" with the datamodel. They are lazy or hurried so
they don't care about possible
keys, about WHERE clauses selectivity and all that kind of things so quick
and dirty solution to be sure to get unique rows : SELECT DISTINCT which
implies a sort therefore more activity to get the same result as they may
have gotten quicker with a little bit work on the WHERE clause.

Alkos
Cris Carampa - 28 Nov 2003 11:05 GMT
> Most of the time, people use SELECT DISTINCT to get unique rows because
> they don't want to "bother" with the datamodel. They are lazy or hurried so
[quoted text clipped - 3 lines]
> implies a sort therefore more activity to get the same result as they may
> have gotten quicker with a little bit work on the WHERE clause.

I do use SELECT DISTINCT when I have just to pick up distinct values
from a table, and I don't need to count them or performing group
functions amongs columns in the table. I assume the performance of this
statement:

SELECT DISTINCT FOOCOL FROM FOOTAB ;

is better than the performance of this one:

SELECT FOOCOL, COUNT(FOOCOL) FROM FOOTAB GROUP BY FOOCOL ;

Or am I wrong?

Signature

Cris Carampa (spamto:cris119@operamail.com)

"Poveri fanatici comunisti, noglobal e affetti dalla sindrome
anti-microsoft" (gli utenti Linux secondo un poster di ICOD)

Frank - 28 Nov 2003 11:46 GMT
>> Most of the time, people use SELECT DISTINCT to get unique rows because
>> they don't want to "bother" with the datamodel. They are lazy or
[quoted text clipped - 18 lines]
>
> Or am I wrong?

Consider

select distinct e.first_name
from employees e, departments d;

versus

select e.first_name
from employees e, departments d
where e.department_id=d.department_id;

Oversimplified, of course, but it's the idea.
Signature

Regards, Frank van Bortel

Daniel Morgan - 28 Nov 2003 18:41 GMT
>>> Most of the time, people use SELECT DISTINCT to get unique rows because
>>> they don't want to "bother" with the datamodel. They are lazy or
[quoted text clipped - 32 lines]
>
> Oversimplified, of course, but it's the idea.

I see no relationship between your two queries.

The first one is a Cartesian join returning distinct first names. The
second is an inner join that returns all first names. They do completely
different things.

Signature

Daniel Morgan
http://www.outreach.washington.edu/ext/certificates/oad/oad_crs.asp
http://www.outreach.washington.edu/ext/certificates/aoa/aoa_crs.asp
damorgan@x.washington.edu
(replace 'x' with a 'u' to reply)

Jeff - 01 Dec 2003 14:22 GMT
>I see no relationship between your two queries.

Neither do I.  He must have some real "winners" working with him if that's the
difference between a "right" and "wrong" query.  ;-)
Alkos - 01 Dec 2003 09:07 GMT
> > Most of the time, people use SELECT DISTINCT to get unique rows because
> > they don't want to "bother" with the datamodel. They are lazy or hurried so
[quoted text clipped - 22 lines]
> "Poveri fanatici comunisti, noglobal e affetti dalla sindrome
> anti-microsoft" (gli utenti Linux secondo un poster di ICOD)

Neither right nor wrong.
Those 2 queries are different in purpose.

select distinct should only be used when a natural key doesn't exist to get
unique rows. This situation is *always* the consequence of a poor datamodel
design.

I use personnaly select distinct when i want to test the cardinality of a
column :
select count(*), count(distinct my col) from bla bla

HTH
Alkos.
Marc Blum - 28 Nov 2003 16:40 GMT
>> (1)
>> SELECT DISTINCT ...
>
>Sorry for the naive question: what's wrong with SELECT DISTINCT?
>
>Kind regards,

DISTINCT requieres a sort. Unnecessary sorts are main reason fpr performance
problems.

In 100% of the cases, where I stumbled about a SELECT DISTINCT, it was coded for
one of these three reasons:

(1) the developer didn't understand the datamodel

(2) the developer didn't understand the requierements he was implementing

(3) the datamodel was inappropriate to the requierements

Over the last three or four years, no single DISTINCT survided, after I
encountered it.

--
Marc Blum
mailto:blumNOSPAM@marcblum.de
http://www.marcblum.de
Daniel Morgan - 28 Nov 2003 18:21 GMT
>> (1) SELECT DISTINCT ...
>
> Sorry for the naive question: what's wrong with SELECT DISTINCT?
>
> Kind regards,

Nothing when used appropriately. It has a cost associated with it and
thus should only be used when it is required to produce the correct result.

Signature

Daniel Morgan
http://www.outreach.washington.edu/ext/certificates/oad/oad_crs.asp
http://www.outreach.washington.edu/ext/certificates/aoa/aoa_crs.asp
damorgan@x.washington.edu
(replace 'x' with a 'u' to reply)

Alkos - 28 Nov 2003 09:52 GMT
> (1)
> SELECT DISTINCT ...
[quoted text clipped - 17 lines]
> mailto:blumNOSPAM@marcblum.de
> http://www.marcblum.de

Hello,

Why
(3)
LOOP
  EXECUTE IMMEDIATE some_sql;
END LOOP;
should be avoided ??

OK, it's Dynamical SQL  but it could be useful to write scalable batch
scripts.
Tell me if i'm wrong.

Cheers,
Alkos
Holger Baer - 28 Nov 2003 11:00 GMT
> Hello,
>
[quoted text clipped - 11 lines]
> Cheers,
> Alkos

If you want it truely scalable, then you would do something like this
(Pseudo code following, so don't trash me for inaccuracies or not
existing function/procedures - it's just supposed to give the
general idea)

dbms_sql.open_cursor
dbms_sql.parse
LOOP
  dbms_sql.bind
  dbms_sql.execute
END LOOP
dbms_sql.close_cursor

This way, you avoid soft parses (and why you should avoid them is
something that you will know if you hang around Tom Kytes website
for a few days at http://asktom.oracle.com )

Personally, I use execute immediate only for administrative tasks.
But then, I'm most of the time the DBA ;-)

Cheers,

Holger
Alkos - 28 Nov 2003 11:07 GMT
> > Hello,
> >
[quoted text clipped - 35 lines]
>
> Holger

Hi Holger,

Ok, the advice should be :
"don't use execute immediate within loops to execute DML queries"*
what i understood first. I was wondering whether there was any hindrance
using exec immdt for DDL in some cases. According to me, there is not
example :
open a cursor to list all tables owned by the current user.
loop throuh the cursor
   exec immdt to grant another user with s,i,u,d on this table
end loop
(this is really pseudo-code,isn't it ? ;-)

Alkos
Holger Baer - 28 Nov 2003 13:03 GMT
>>If you want it truely scalable, then you would do something like this
>>(Pseudo code following, so don't trash me for inaccuracies or not
[quoted text clipped - 24 lines]
> Ok, the advice should be :
> "don't use execute immediate within loops to execute DML queries"*

That is exactly right.

> what i understood first. I was wondering whether there was any hindrance
> using exec immdt for DDL in some cases. According to me, there is not
[quoted text clipped - 4 lines]
> end loop
> (this is really pseudo-code,isn't it ? ;-)

As pseudo as can be, eh? ;-)

But you're right, that's the way to use execute immediate, at least,
that's what I use it for.

Cheers

Holger
 
Sign In
Join
My Latest Posts
My Monitored Threads
My Blog
My Photo Gallery
My Profile
My Homepage

Start New Thread
Enable EMail Alerts
Rate this Thread



©2009 Advenet LLC   Privacy Policy - Terms of Use
This website includes both content owned or controlled by Advenet as well as content owned or controlled by third parties.