Home | Contact Us | FAQ | Search & Site Map | Link to Us
Sign In | Join | Other 45 Sites in Network
Home
Discussion Groups
Database Servers
DB2InformixIngresMS SQLOraclePervasive.SQLPostgreSQLProgressSybase
Desktop Databases
FileMakerFoxProMS AccessParadox
General
General DB TopicsDatabase Theory
Related Topics
Java Development.NET DevelopmentVB DevelopmentMore Topics ...

Database Forum / General DB Topics / General DB Topics / November 2004

Tip: Looking for answers? Try searching our database.

version controle tools

Thread view: 
Enable EMail Alerts  Start New Thread
Thread rating: 
satheeskumar - 15 Nov 2004 17:54 GMT
..
Are there any tools ?
The version controle system in microsoft product is vss.In oracle is
cvs.In db/2 what they use?,informix..?.

Question?
Are there any common IDE for the above databases like css or vss to
controle the version between different data bases?.

rgds
satheeskumar
Jeff Clausius - 16 Nov 2004 16:19 GMT
satheeskumar:

I've worked in Informix, Oracle, and SQL Server.  In all cases, the
enviornments for controlling DDL is lacking when it comes to version
control, which is extremely important for controlling your tables, stored
procs, etc.

In the end ( for each platform ), I ended up creating scripts for the
entire DDL, and then placing the scripts within a version control tool.  

In fact, that is exactly how things are done for the SourceGear Vault
version control tool  ( OK, shameless plug )

HTH
Jeff Clausius
SourceGear

> ..
> Are there any tools ?
[quoted text clipped - 7 lines]
> rgds
> satheeskumar
Christopher Browne - 16 Nov 2004 19:12 GMT
In an attempt to throw the authorities off his trail, Jeff Clausius <jeffc@raegecruos.com> transmitted:
> satheeskumar:
>
[quoted text clipped - 12 lines]
> Jeff Clausius
> SourceGear

That doesn't resolve how to make incremental changes to database
schema...

Supposing we add 4 fields to a table, the "diff" arguably involves
invoking a bunch of ALTER TABLE requests.
Signature

let name="cbbrowne" and tld="linuxfinances.info" in String.concat "@" [name;tld];;
http://linuxfinances.info/info/nonrdbms.html
"For every bad idea you can find a crapplication using it.  Nothing new
about that..." -- Alexander Viro

Kenneth Downs - 17 Nov 2004 02:09 GMT
> In an attempt to throw the authorities off his trail, Jeff Clausius
> <jeffc@raegecruos.com> transmitted:
[quoted text clipped - 20 lines]
> Supposing we add 4 fields to a table, the "diff" arguably involves
> invoking a bunch of ALTER TABLE requests.

Christopher, may I ask you handle such things now?  I have my own ideas, but
I'm curious as to what you may offer on the subject.

Signature

Kenneth Downs

java.lang.String.tcpip.usenet.sigblock.setSig("After finally finding the
object and method to set the sigblock, I forgot what I wanted to say!");

Christopher Browne - 17 Nov 2004 21:22 GMT
Centuries ago, Nostradamus foresaw when Kenneth Downs <firstinit.lastname@lastnameplusfam.net> would write:

>> In an attempt to throw the authorities off his trail, Jeff Clausius
>> <jeffc@raegecruos.com> transmitted:
[quoted text clipped - 23 lines]
> Christopher, may I ask you handle such things now?  I have my own ideas, but
> I'm curious as to what you may offer on the subject.

Actually, I have seen a bit of "movement" recently.

What I have is _highly_ specific to PostgreSQL, but I'll bet the
approach could be used more widely:

Here's a stored procedure that checks to see if a particular table,
"sl_node", has been modified to include the new field, "no_spool", and
adds it, if necessary:

create or replace function @NAMESPACE@.upgrade_sl_node () returns bool
as '
DECLARE
       v_row  record;
BEGIN
  select 1 into v_row
    from pg_namespace n, pg_class c, pg_attribute a
    where
       n.nspname = ''@NAMESPACE@'' and
       c.relnamespace = n.oid and
       c.relname = ''sl_node'' and
       a.attrelid = c.oid and
       a.attname = ''no_spool'';
  if not found then
       raise notice ''Upgrade sl_node - add field no_spool'';
       alter table @NAMESPACE@.sl_node
           add column no_spool boolean default ''f'';
       update @NAMESPACE@.sl_node set no_spool = ''f'';
       return ''t'';
  end if;
  return ''f'';
END;' language plpgsql;

I have already written a procedure that generalizes this a bit
further; you get to do column-by-column updates by something like the
following:

 select add_missing_column('schema1', 'table_x', 'field_1', 'integer');
 select add_missing_column('schema1', 'table_x', 'field_2', 'numeric(20,4)');
 select add_missing_column('schema1', 'table_x', 'field_3', 'character(4)');

If you started off with enough of the table to at least have the
primary key, you could write a query that would pull out all of the
necessary bits of metadata from a database, something like...

select nspname as schema, relname as table_name, attname as
 field_name, [something indicating type that I haven't thought thru yet]
into exportable_metadata
from [various tables]
where
 [joining needful bits]
 [restricting to interesting tables];

Export that table, and then run the query:

select add_missing_column(schema, table_name, field_name, field_type)
  from exportable_metadata;

It's not directly portable, but other databases that have the ANSI
standard information schema may be able to do analagous things.

I think it's a plenty useful approach.  It copes only with differences
from one version to another; still useful.
Signature

(reverse (concatenate 'string "gro.gultn" "@" "enworbbc"))
http://linuxfinances.info/info/postgresql.html
Eagles may soar, free and proud, but weasels never get sucked into jet
engines.

Kenneth Downs - 18 Nov 2004 12:03 GMT
> Centuries ago, Nostradamus foresaw when Kenneth Downs
> <firstinit.lastname@lastnameplusfam.net> would write:
[quoted text clipped - 42 lines]
>   'numeric(20,4)'); select add_missing_column('schema1', 'table_x',
>   'field_3', 'character(4)');

Hmmm.  Do you generate the column commands from some outside source of
information?  

You may find it easier to deal with the INFORMATION_SCHEMA tables, one of
which is INFORMATION_SCHEMA.columns.  This gives you a nice table of the
columns in any table.  Then you can do a simple select:

SELECT * from MyColumnDefs
 WHERE column_id not in
 (Select column_name from Information_Schema...)

This is now the list of columns to create.

AFAIK, Postgres requires one ALTER command per column, which is a bummer.

> If you started off with enough of the table to at least have the
> primary key, you could write a query that would pull out all of the
[quoted text clipped - 15 lines]
> It's not directly portable, but other databases that have the ANSI
> standard information schema may be able to do analagous things.

I was able to make the process highly portable by putting the code
client-side and having it generate the ALTER commands.  This reduces the
platform-specific stuff to just querying the server to determine current
status, which you can do either from platform specific tables like the pg_*
stuff, or from the INFORMATION_SCHEMA stuff, which is SQL standard.

> I think it's a plenty useful approach.  It copes only with differences
> from one version to another; still useful.

Actually you may have more here than you realize.  If you supply it with an
outside reference, basically saying "here, build these tables", then you
can go from any version to any other version without going through each
version inbetween.

Signature

Kenneth Downs

java.lang.String.tcpip.usenet.posting.response.sigblock.setSig("After
finally finding the right object abd method to set the sig block, I forgot
what I wanted to say!");

Jeff Clausius - 18 Nov 2004 15:18 GMT
Christopher:

You are correct.  No one asked about that.

To answer your question, I can think of a couple different ways to
achieve this goal.  I'm providing two methods that I've used in the past.  
There may be other solutions out there that others would like to
contribute.

Method 1 - Upgrade scripts - Along with the DDL describing the whole
schema, I keep an "upgrade" script which contains the DDL to add in the
differences between change to change.  So, when I make a change to a
table, I also create the ALTER statement that would change the table.  
This goes into the upgrade script.  There is more to it then that, but
you should get the general idea.  This model requires quite a bit of
hands on work, but with something as important as a database schema, I
feel this gives the best control.

Method 2 - SQL Diff Tool - There are SQL diff tools that exist.  I don't
believe any of them can diff two script files, but they can diff two
databases and generate the DDL to merge the changes.  These tools could
be used to generate the differences of databases generated from scripts
kept in a version control tool.

Jeff

> That doesn't resolve how to make incremental changes to database
> schema...
>
> Supposing we add 4 fields to a table, the "diff" arguably involves
> invoking a bunch of ALTER TABLE requests.
Kenneth Downs - 18 Nov 2004 23:34 GMT
> Christopher:
>
[quoted text clipped - 13 lines]
> hands on work, but with something as important as a database schema, I
> feel this gives the best control.

In my experience these do not scale at all, either in script count or
programmer count.  The problem is that you have one-time code running at
the highest privilege level making many many assumptions about the prior
state of its target.  If you start trying to pull features out of sequence
it becomes a nightmare.

In a high volume situation, assume each script has a 99% chance of success,
which is extremely optimistic.  Two scripts together have 99 x 99 = ~98%,
three have 99^3 = ~97 and so forth.  The more scripts the less chance that
it will work.

There is no way out either.  The fact that the scripts are running at the
highest privilege level makes all enforcement techniques useless, a
programmer under deadline pressure will put anything he thinks will work
into the script, and you do not want to be the person saying the product
should not ship because of his script.

> Method 2 - SQL Diff Tool - There are SQL diff tools that exist.  I don't
> believe any of them can diff two script files, but they can diff two
                              ^^^^^^^^^^^^^^

The thing to do is diff two specs defined in tables.  Basically you want
stuff like the following pseudo-query:

SELECT * FROM Spec_Columns
 WHERE column_ID NOT IN
 (Select Column_id from CurrentState_Columns)

...the next step is to build DDL to alter tables and add those columns.

> databases and generate the DDL to merge the changes.  These tools could
> be used to generate the differences of databases generated from scripts
> kept in a version control tool.

I've always built database builders of one kind or another, and one
particularly ambitious one diff'd all server-side elements, including null
rules, defaults, stored procedures, functions, triggers and other stuff.
That experience convinced me that deliverying code to the server via a diff
tool is a mis-match, all server-side code should be generated out of scalar
data.  This is the only way to prevent programmers from violating
conventions and best practices policies.

My current project is a purely data-driven database builder, if you are
interested in updates on that let me know.

> Jeff
>
[quoted text clipped - 3 lines]
>> Supposing we add 4 fields to a table, the "diff" arguably involves
>> invoking a bunch of ALTER TABLE requests.

Signature

Kenneth Downs

java.lang.String.tcpip.usenet.posting.response.sigblock.setSig("After
finally finding the right object abd method to set the sig block, I forgot
what I wanted to say!");

Jeff Clausius - 23 Nov 2004 15:30 GMT
Kenneth:

Are we talking about SQL schemas or deployment?  The main topic of this
thread was keeping track of changes made to schemas, not deployment.

However, in response to your assessment to deployment, I disagree.  I've
done this for about 9 distinct databases ( probably about 30 to 40 versions
) over 12 years and have not had one failure.  The success rate so far is
100%.

I think your getting confused on the scripting issue.  I only use 1 upgrade
script.  The file has error checking throughout the entire file, so if one
of the conditions is not met, the script stops and reports the error.  

They key is not making any assumptions about the state of the schema, and
testing ALL conditions to know what a table looks like before running any
DDL.  Again, this is very, very tedious work, but the solution, if handled
properly does work.

The important thing to remember is the schema is delicate, and should be
handled with care.  As you mention, any developer who rushes things can
cause problems.

With that said, I'd be interested in furthering the discussion on the
subject of DB deployment.

Jeff Clausius

>> Christopher:
>>
[quoted text clipped - 68 lines]
>>> Supposing we add 4 fields to a table, the "diff" arguably involves
>>> invoking a bunch of ALTER TABLE requests.
Kenneth Downs - 23 Nov 2004 20:56 GMT
> Kenneth:
>
> Are we talking about SQL schemas or deployment?  The main topic of this
> thread was keeping track of changes made to schemas, not deployment.

See below, I believe they are the same.

> However, in response to your assessment to deployment, I disagree.  I've
> done this for about 9 distinct databases ( probably about 30 to 40
> versions
> ) over 12 years and have not had one failure.  The success rate so far is
> 100%.

Then you should stick with it.

> With that said, I'd be interested in furthering the discussion on the
> subject of DB deployment.
>
> Jeff Clausius

From my point of view, which has always been that of the ISV, deployment of
a complete application is:

1)  Installing and updating schemas, including server-side code like
   triggers, views, etc.

2)  Populating system tables

3)  Copying in code

From this point of view, a conversation about schemas is always a subset of
a conversation about deployment.  In the largest context, it is all about
packaging, delivering and installing the three items listed above.

From here you can see that regular development is a lot about deployment.  A
programmer wants to work out something, so he does any one or all of the
three things: schema, content, code.  When it works well he wants it in the
QA system for regression and integration tests.  Another run of schema,
content, and code.  Then it goes to other test stations, and finally into
release.  Then it goes to the customer's test system, finally into live.
Then all of the customers want it.  Every time it is schema, content, and
code, over and over.  That is a lot of deployment.

Then this crucial feature must be back-ported to a customer 6 months behind
who does not want the disruption of a major upgrade.  So we need a reliable
way to do schema, content and code to that back system.  A lot of
deployment.

I am aware that many people do not agree with this, but I like it for the
same reason you like your upgrade scripts.

Signature

Kenneth Downs

java.lang.String.tcpip.usenet.posting.response.sigblock.setSig("After
finally finding the right object abd method to set the sig block, I forgot
what I wanted to say!");

Jeff Clausius - 23 Nov 2004 22:05 GMT
> Then you should stick with it.
Yes, so far, so good.

> From my point of view, which has always been that of the ISV,
> deployment of a complete application is:
[quoted text clipped - 28 lines]
> I am aware that many people do not agree with this, but I like it for
> the same reason you like your upgrade scripts.

Yes.  The trick is to design a system capable of updating a schema which
is 1 day old, 3 weeks old, 6 months old, or on Version 0.1.  It sounds
like your DB builder accomplishes this same task as my script based
solution.  Very cool.

Jeff
 
Sign In
Join
My Latest Posts
My Monitored Threads
My Blog
My Photo Gallery
My Profile
My Homepage

Start New Thread
Enable EMail Alerts
Rate this Thread



©2009 Advenet LLC   Privacy Policy - Terms of Use
This website includes both content owned or controlled by Advenet as well as content owned or controlled by third parties.