Database Forum / General DB Topics / General DB Topics / November 2004
version controle tools
|
|
Thread rating:  |
satheeskumar - 15 Nov 2004 17:54 GMT .. Are there any tools ? The version controle system in microsoft product is vss.In oracle is cvs.In db/2 what they use?,informix..?.
Question? Are there any common IDE for the above databases like css or vss to controle the version between different data bases?.
rgds satheeskumar
Jeff Clausius - 16 Nov 2004 16:19 GMT satheeskumar:
I've worked in Informix, Oracle, and SQL Server. In all cases, the enviornments for controlling DDL is lacking when it comes to version control, which is extremely important for controlling your tables, stored procs, etc.
In the end ( for each platform ), I ended up creating scripts for the entire DDL, and then placing the scripts within a version control tool.
In fact, that is exactly how things are done for the SourceGear Vault version control tool ( OK, shameless plug )
HTH Jeff Clausius SourceGear
> .. > Are there any tools ? [quoted text clipped - 7 lines] > rgds > satheeskumar Christopher Browne - 16 Nov 2004 19:12 GMT In an attempt to throw the authorities off his trail, Jeff Clausius <jeffc@raegecruos.com> transmitted:
> satheeskumar: > [quoted text clipped - 12 lines] > Jeff Clausius > SourceGear That doesn't resolve how to make incremental changes to database schema...
Supposing we add 4 fields to a table, the "diff" arguably involves invoking a bunch of ALTER TABLE requests.
 Signature let name="cbbrowne" and tld="linuxfinances.info" in String.concat "@" [name;tld];; http://linuxfinances.info/info/nonrdbms.html "For every bad idea you can find a crapplication using it. Nothing new about that..." -- Alexander Viro
Kenneth Downs - 17 Nov 2004 02:09 GMT > In an attempt to throw the authorities off his trail, Jeff Clausius > <jeffc@raegecruos.com> transmitted: [quoted text clipped - 20 lines] > Supposing we add 4 fields to a table, the "diff" arguably involves > invoking a bunch of ALTER TABLE requests. Christopher, may I ask you handle such things now? I have my own ideas, but I'm curious as to what you may offer on the subject.
 Signature Kenneth Downs
java.lang.String.tcpip.usenet.sigblock.setSig("After finally finding the object and method to set the sigblock, I forgot what I wanted to say!");
Christopher Browne - 17 Nov 2004 21:22 GMT Centuries ago, Nostradamus foresaw when Kenneth Downs <firstinit.lastname@lastnameplusfam.net> would write:
>> In an attempt to throw the authorities off his trail, Jeff Clausius >> <jeffc@raegecruos.com> transmitted: [quoted text clipped - 23 lines] > Christopher, may I ask you handle such things now? I have my own ideas, but > I'm curious as to what you may offer on the subject. Actually, I have seen a bit of "movement" recently.
What I have is _highly_ specific to PostgreSQL, but I'll bet the approach could be used more widely:
Here's a stored procedure that checks to see if a particular table, "sl_node", has been modified to include the new field, "no_spool", and adds it, if necessary:
create or replace function @NAMESPACE@.upgrade_sl_node () returns bool as ' DECLARE v_row record; BEGIN select 1 into v_row from pg_namespace n, pg_class c, pg_attribute a where n.nspname = ''@NAMESPACE@'' and c.relnamespace = n.oid and c.relname = ''sl_node'' and a.attrelid = c.oid and a.attname = ''no_spool''; if not found then raise notice ''Upgrade sl_node - add field no_spool''; alter table @NAMESPACE@.sl_node add column no_spool boolean default ''f''; update @NAMESPACE@.sl_node set no_spool = ''f''; return ''t''; end if; return ''f''; END;' language plpgsql;
I have already written a procedure that generalizes this a bit further; you get to do column-by-column updates by something like the following:
select add_missing_column('schema1', 'table_x', 'field_1', 'integer'); select add_missing_column('schema1', 'table_x', 'field_2', 'numeric(20,4)'); select add_missing_column('schema1', 'table_x', 'field_3', 'character(4)');
If you started off with enough of the table to at least have the primary key, you could write a query that would pull out all of the necessary bits of metadata from a database, something like...
select nspname as schema, relname as table_name, attname as field_name, [something indicating type that I haven't thought thru yet] into exportable_metadata from [various tables] where [joining needful bits] [restricting to interesting tables];
Export that table, and then run the query:
select add_missing_column(schema, table_name, field_name, field_type) from exportable_metadata;
It's not directly portable, but other databases that have the ANSI standard information schema may be able to do analagous things.
I think it's a plenty useful approach. It copes only with differences from one version to another; still useful.
 Signature (reverse (concatenate 'string "gro.gultn" "@" "enworbbc")) http://linuxfinances.info/info/postgresql.html Eagles may soar, free and proud, but weasels never get sucked into jet engines.
Kenneth Downs - 18 Nov 2004 12:03 GMT > Centuries ago, Nostradamus foresaw when Kenneth Downs > <firstinit.lastname@lastnameplusfam.net> would write: [quoted text clipped - 42 lines] > 'numeric(20,4)'); select add_missing_column('schema1', 'table_x', > 'field_3', 'character(4)'); Hmmm. Do you generate the column commands from some outside source of information?
You may find it easier to deal with the INFORMATION_SCHEMA tables, one of which is INFORMATION_SCHEMA.columns. This gives you a nice table of the columns in any table. Then you can do a simple select:
SELECT * from MyColumnDefs WHERE column_id not in (Select column_name from Information_Schema...)
This is now the list of columns to create.
AFAIK, Postgres requires one ALTER command per column, which is a bummer.
> If you started off with enough of the table to at least have the > primary key, you could write a query that would pull out all of the [quoted text clipped - 15 lines] > It's not directly portable, but other databases that have the ANSI > standard information schema may be able to do analagous things. I was able to make the process highly portable by putting the code client-side and having it generate the ALTER commands. This reduces the platform-specific stuff to just querying the server to determine current status, which you can do either from platform specific tables like the pg_* stuff, or from the INFORMATION_SCHEMA stuff, which is SQL standard.
> I think it's a plenty useful approach. It copes only with differences > from one version to another; still useful. Actually you may have more here than you realize. If you supply it with an outside reference, basically saying "here, build these tables", then you can go from any version to any other version without going through each version inbetween.
 Signature Kenneth Downs
java.lang.String.tcpip.usenet.posting.response.sigblock.setSig("After finally finding the right object abd method to set the sig block, I forgot what I wanted to say!");
Jeff Clausius - 18 Nov 2004 15:18 GMT Christopher:
You are correct. No one asked about that.
To answer your question, I can think of a couple different ways to achieve this goal. I'm providing two methods that I've used in the past. There may be other solutions out there that others would like to contribute.
Method 1 - Upgrade scripts - Along with the DDL describing the whole schema, I keep an "upgrade" script which contains the DDL to add in the differences between change to change. So, when I make a change to a table, I also create the ALTER statement that would change the table. This goes into the upgrade script. There is more to it then that, but you should get the general idea. This model requires quite a bit of hands on work, but with something as important as a database schema, I feel this gives the best control.
Method 2 - SQL Diff Tool - There are SQL diff tools that exist. I don't believe any of them can diff two script files, but they can diff two databases and generate the DDL to merge the changes. These tools could be used to generate the differences of databases generated from scripts kept in a version control tool.
Jeff
> That doesn't resolve how to make incremental changes to database > schema... > > Supposing we add 4 fields to a table, the "diff" arguably involves > invoking a bunch of ALTER TABLE requests. Kenneth Downs - 18 Nov 2004 23:34 GMT > Christopher: > [quoted text clipped - 13 lines] > hands on work, but with something as important as a database schema, I > feel this gives the best control. In my experience these do not scale at all, either in script count or programmer count. The problem is that you have one-time code running at the highest privilege level making many many assumptions about the prior state of its target. If you start trying to pull features out of sequence it becomes a nightmare.
In a high volume situation, assume each script has a 99% chance of success, which is extremely optimistic. Two scripts together have 99 x 99 = ~98%, three have 99^3 = ~97 and so forth. The more scripts the less chance that it will work.
There is no way out either. The fact that the scripts are running at the highest privilege level makes all enforcement techniques useless, a programmer under deadline pressure will put anything he thinks will work into the script, and you do not want to be the person saying the product should not ship because of his script.
> Method 2 - SQL Diff Tool - There are SQL diff tools that exist. I don't > believe any of them can diff two script files, but they can diff two ^^^^^^^^^^^^^^
The thing to do is diff two specs defined in tables. Basically you want stuff like the following pseudo-query:
SELECT * FROM Spec_Columns WHERE column_ID NOT IN (Select Column_id from CurrentState_Columns)
...the next step is to build DDL to alter tables and add those columns.
> databases and generate the DDL to merge the changes. These tools could > be used to generate the differences of databases generated from scripts > kept in a version control tool. I've always built database builders of one kind or another, and one particularly ambitious one diff'd all server-side elements, including null rules, defaults, stored procedures, functions, triggers and other stuff. That experience convinced me that deliverying code to the server via a diff tool is a mis-match, all server-side code should be generated out of scalar data. This is the only way to prevent programmers from violating conventions and best practices policies.
My current project is a purely data-driven database builder, if you are interested in updates on that let me know.
> Jeff > [quoted text clipped - 3 lines] >> Supposing we add 4 fields to a table, the "diff" arguably involves >> invoking a bunch of ALTER TABLE requests.
 Signature Kenneth Downs
java.lang.String.tcpip.usenet.posting.response.sigblock.setSig("After finally finding the right object abd method to set the sig block, I forgot what I wanted to say!");
Jeff Clausius - 23 Nov 2004 15:30 GMT Kenneth:
Are we talking about SQL schemas or deployment? The main topic of this thread was keeping track of changes made to schemas, not deployment.
However, in response to your assessment to deployment, I disagree. I've done this for about 9 distinct databases ( probably about 30 to 40 versions ) over 12 years and have not had one failure. The success rate so far is 100%.
I think your getting confused on the scripting issue. I only use 1 upgrade script. The file has error checking throughout the entire file, so if one of the conditions is not met, the script stops and reports the error.
They key is not making any assumptions about the state of the schema, and testing ALL conditions to know what a table looks like before running any DDL. Again, this is very, very tedious work, but the solution, if handled properly does work.
The important thing to remember is the schema is delicate, and should be handled with care. As you mention, any developer who rushes things can cause problems.
With that said, I'd be interested in furthering the discussion on the subject of DB deployment.
Jeff Clausius
>> Christopher: >> [quoted text clipped - 68 lines] >>> Supposing we add 4 fields to a table, the "diff" arguably involves >>> invoking a bunch of ALTER TABLE requests. Kenneth Downs - 23 Nov 2004 20:56 GMT > Kenneth: > > Are we talking about SQL schemas or deployment? The main topic of this > thread was keeping track of changes made to schemas, not deployment. See below, I believe they are the same.
> However, in response to your assessment to deployment, I disagree. I've > done this for about 9 distinct databases ( probably about 30 to 40 > versions > ) over 12 years and have not had one failure. The success rate so far is > 100%. Then you should stick with it.
> With that said, I'd be interested in furthering the discussion on the > subject of DB deployment. > > Jeff Clausius From my point of view, which has always been that of the ISV, deployment of a complete application is:
1) Installing and updating schemas, including server-side code like triggers, views, etc.
2) Populating system tables
3) Copying in code
From this point of view, a conversation about schemas is always a subset of a conversation about deployment. In the largest context, it is all about packaging, delivering and installing the three items listed above.
From here you can see that regular development is a lot about deployment. A programmer wants to work out something, so he does any one or all of the three things: schema, content, code. When it works well he wants it in the QA system for regression and integration tests. Another run of schema, content, and code. Then it goes to other test stations, and finally into release. Then it goes to the customer's test system, finally into live. Then all of the customers want it. Every time it is schema, content, and code, over and over. That is a lot of deployment.
Then this crucial feature must be back-ported to a customer 6 months behind who does not want the disruption of a major upgrade. So we need a reliable way to do schema, content and code to that back system. A lot of deployment.
I am aware that many people do not agree with this, but I like it for the same reason you like your upgrade scripts.
 Signature Kenneth Downs
java.lang.String.tcpip.usenet.posting.response.sigblock.setSig("After finally finding the right object abd method to set the sig block, I forgot what I wanted to say!");
Jeff Clausius - 23 Nov 2004 22:05 GMT > Then you should stick with it. Yes, so far, so good.
> From my point of view, which has always been that of the ISV, > deployment of a complete application is: [quoted text clipped - 28 lines] > I am aware that many people do not agree with this, but I like it for > the same reason you like your upgrade scripts. Yes. The trick is to design a system capable of updating a schema which is 1 day old, 3 weeks old, 6 months old, or on Version 0.1. It sounds like your DB builder accomplishes this same task as my script based solution. Very cool.
Jeff
|
|
|